[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

    RE: iSCSI/iWARP drafts and flow control

    -----Original Message-----
    From: Caitlin Bestler []
    Sent: Thursday, July 31, 2003 10:59 AM
    Subject: Re: iSCSI/iWARP drafts and flow control
    On Thursday, July 31, 2003, at 12:30 PM, wrote:
    > Caitlin,
    > There are multiple problems with the mechanism you suggest.
    > For one thing, the credit for additional command window in iSCSI isn't 
    > based on command replys. There is an explicit field for MaxCmdSN. 
    > Having received and finished processing a command doesn't mean that 
    > one has made space for receiving another command. The target might 
    > choose to use that space for another session or for something else 
    > altogether.
    There is a restocking mechanism in place. I oversimplified in my 
    description of it. The point is that
    the ULP already knows how many CmdSNs that it is allowed to send. It 
    knows when that count is depleted,
    and when it is restored.
    > Secondly, having finished a processing a command doesn't mean that 
    > other activities have completed. If we did what you suggest there 
    > might be times when a response had to be delayed because the command 
    > had been finished but the credits for the non-command messages were 
    > not yet ready. This could adversely effect performance. Also, commands 
    > can involve lengthy actions such as large data transfers. Therefore, 
    > there might be times when one is not ready to send a command reply and 
    > immediate commands or other non-command related traffic are delayed 
    > because of lack of credit.
    There is no requirement that the asynch processing have completed, just 
    that the buffers have been restocked.
    If the number of asynch commands is light, as claimed, crediting enough 
    to cover the delay of a long command
    should not be a problem. The goal is to grant credits, nobody is 
    requiring buffers be pre-committed for the
    entire lifespan of the credit.
    > Third, it is an accounting nightmare for the implementations. Instead 
    > of just tracking a simple credit counter and updating it, they have to 
    > keep track of how many credits belong to each command.
    A nightmare? At worst it is a second count.
    One implementation could be as simple as setting a flag that will be 
    cleared after N cmds.
    > Fourth, the suggestion only takes care of one direction. MaxCmdSN goes 
    > from target to initiator, but there are types of PDUs not covered by 
    > that mechanisms that go each direction.
    Then how are they being flow controlled?
    > Fifth, it may have have a deadlock. If the command window has closed, 
    > the only way the target can send more credit is to send a NOP PDU. If 
    > the target doesn't have a credit, it won't be able to send the NOP > PDU.
    How is this prevented with iSCSI over TCP?
    What iWARP is requiring is that the ULP provide flow control for 
    untagged buffers,
    to replace the flow control that TCP/SCTP provided for transport 
    This is a *benefit*, it prevents head-of-line blocking of tagged 
    transfers. But
    in any situation, if the ULP could have a deadlock situation where 
    there are
    no untagged buffers available at either end then TCP/SCTP could have 
    had a
    deadlock on transport buffering.
    > It isn't a good idea to complicate implementations excessively to 
    > avoid additions to wire protocol. If one adds a flow control 
    > mechanism, it should have provision to avoid deadlock.
    > Pat
    Meanwhile you have not addressed any of the critical problems with the 
    lack of
    ULP flow control.
    Without flow control the sender must *guess* at how many messages can 
    be sent.
    If they guess wrong the stream is terminated. Stream termination is NOT
    "flow control", it is either a *sanction* for failing to conform to the
    ULP, or it is the result of a *fault* on the receiving side.
    There is, by definition, no reliable mechanism to "guess" how many 
    may be sent that does not produce stream terminations. That is 
    in something that is supposed to be a reliable protocol.
    If you don't want the semantics of a reliable protocol then propose an
    adaptation that runs over UDP. Otherwise, you cannot have a reliable
    protocol without *some* form of flow control on buffer usage. You cannot
    require the Data Sink to provide an *indefinite* amount of buffering.
    The arguments you cite seem to prove that iSER cannot be implemented 
    a standard RNIC. I believe it can be, but that requires documenting the
    rules that will *regulate* transmission of *all* untagged messages.
    I believe this can be done in a way that meets iSCSI's requirements 
    violating iWARP's rules. If iSCSI works over TCP, where its messages 
    can be
    flow controlled by the TCP window, then it can work over a definition of
    iSER that complies with iWARPs rules and the definition "reliable 


Last updated: Tue Aug 05 12:46:08 2003
12771 messages in chronological order