[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: iSCSI/iWARP drafts and flow control
On Thursday, July 31, 2003, at 12:30 PM, email@example.com wrote: > Caitlin, > > There are multiple problems with the mechanism you suggest. > > For one thing, the credit for additional command window in iSCSI isn't > based on command replys. There is an explicit field for MaxCmdSN. > Having received and finished processing a command doesn't mean that > one has made space for receiving another command. The target might > choose to use that space for another session or for something else > altogether. > There is a restocking mechanism in place. I oversimplified in my description of it. The point is that the ULP already knows how many CmdSNs that it is allowed to send. It knows when that count is depleted, and when it is restored. > Secondly, having finished a processing a command doesn't mean that > other activities have completed. If we did what you suggest there > might be times when a response had to be delayed because the command > had been finished but the credits for the non-command messages were > not yet ready. This could adversely effect performance. Also, commands > can involve lengthy actions such as large data transfers. Therefore, > there might be times when one is not ready to send a command reply and > immediate commands or other non-command related traffic are delayed > because of lack of credit. > There is no requirement that the asynch processing have completed, just that the buffers have been restocked. If the number of asynch commands is light, as claimed, crediting enough to cover the delay of a long command should not be a problem. The goal is to grant credits, nobody is requiring buffers be pre-committed for the entire lifespan of the credit. > Third, it is an accounting nightmare for the implementations. Instead > of just tracking a simple credit counter and updating it, they have to > keep track of how many credits belong to each command. > A nightmare? At worst it is a second count. One implementation could be as simple as setting a flag that will be cleared after N cmds. > Fourth, the suggestion only takes care of one direction. MaxCmdSN goes > from target to initiator, but there are types of PDUs not covered by > that mechanisms that go each direction. > Then how are they being flow controlled? > Fifth, it may have have a deadlock. If the command window has closed, > the only way the target can send more credit is to send a NOP PDU. If > the target doesn't have a credit, it won't be able to send the NOP > PDU. How is this prevented with iSCSI over TCP? What iWARP is requiring is that the ULP provide flow control for untagged buffers, to replace the flow control that TCP/SCTP provided for transport buffers. This is a *benefit*, it prevents head-of-line blocking of tagged transfers. But in any situation, if the ULP could have a deadlock situation where there are no untagged buffers available at either end then TCP/SCTP could have had a deadlock on transport buffering. > > It isn't a good idea to complicate implementations excessively to > avoid additions to wire protocol. If one adds a flow control > mechanism, it should have provision to avoid deadlock. > > Pat Meanwhile you have not addressed any of the critical problems with the lack of ULP flow control. Without flow control the sender must *guess* at how many messages can be sent. If they guess wrong the stream is terminated. Stream termination is NOT "flow control", it is either a *sanction* for failing to conform to the ULP, or it is the result of a *fault* on the receiving side. There is, by definition, no reliable mechanism to "guess" how many messages may be sent that does not produce stream terminations. That is unacceptable in something that is supposed to be a reliable protocol. If you don't want the semantics of a reliable protocol then propose an adaptation that runs over UDP. Otherwise, you cannot have a reliable protocol without *some* form of flow control on buffer usage. You cannot require the Data Sink to provide an *indefinite* amount of buffering. The arguments you cite seem to prove that iSER cannot be implemented over a standard RNIC. I believe it can be, but that requires documenting the rules that will *regulate* transmission of *all* untagged messages. I believe this can be done in a way that meets iSCSI's requirements without violating iWARP's rules. If iSCSI works over TCP, where its messages can be flow controlled by the TCP window, then it can work over a definition of iSER that complies with iWARPs rules and the definition "reliable transport".
Last updated: Tue Aug 05 12:46:08 2003
12771 messages in chronological order