[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

    RE: iSCSI: No Framing

    	Framing of any kind is not a panacea allowing for almost buffering
    free target implementations, it can only move the buffering from the
    transport layer up to the iSCSI layer. Remember that the target MUST
    guarantee commands are delivered to SCSI in the command sequence
    number order. This means if a dropped packet contains an iSCSI command
    PDU you are going to have to buffer until the CmdSN hole is plugged.
    	Consider the example of an initiator filling the command window,
    perhaps sending CmdSNs 1 through 100, possibly along with immediate or
    unsolicited data for some of the commands. If the packet dropped
    contained the command PDU for CmdSN 50 then the target iSCSI layer
    must buffer CmdSNs 51 through 100 and their associated DATA_OUT PDUs.
    Since you can continue to process any DATA_OUT PDUs for CmdSNs 1
    through 49 the buffering required is reduced, but not eliminated,
    	Only if the packet dropped contains DATA_OUTs and no command PDUs can
    you continue processing past the hole while waiting for it to be
    filled. You can R2T for data associated with commands after the hole
    but that serves only to potentially decrease recovery time, not reduce
    buffer requirements.
    	Don't get me wrong, I'm not saying framing isn't a good thing but it
    is only one piece of the puzzle.
    	- Rod
    -----Original Message-----
    From: []On Behalf Of
    THALER,PAT (A-Roseville,ex1)
    Sent: Sunday, February 03, 2002 11:01 PM
    To: WENDT,JIM (HP-Roseville,ex1);
    Subject: RE: iSCSI: No Framing
    In answer to your questions:
    Agilent is planning to implement framing. We view both FIM and
    COWS as having about the same utility so we would implement
    whichever one went into the standard.
    A smoothing buffer on the chip is feasible wrt your point 2.
    There are some configurations that would use external memory.
    Also, one concern is that the very situation where one would
    need large window size for efficiency - high bandwidth long
    distance communication - is where out of order receipt becomes
    increasingly likely so the amount of memory for this situation
    could push up cost to an excessive extent.
    Reducing the amount of traffic that has to be shunted to an
    external memory affects the bandwidth that needs to be provided
    for that memory. If we can handle most PDUs internally then the
    external memory doesn't have to be as wide and as fast. For
    instance, if a drop means that the iSCSI PDU with the drop
    is put into external memory then that takes less memory bandwidth
    than if a drop means the whole data stream for that session will
    be going into the buffer until the hole is filled.
    This decision also effects latency after a drop and required
    backplane bandwidth.
    Let's say one is on a 10 Gbit/s extreme long distance WAN
    connection and that there is a drop. If the round trip delay
    for getting the whole filled is 100 ms, then without framing,
    one could have about 1 Gbit stored in the local buffer memory
    by the time the hole is filled. One will have to process all
    of this through iSCSI and across the backplane before one can
    deal with the new traffic coming in. If the backplane data rate
    is closely sized to the external data rate, an extra 100 ms latency
    has just been inserted into the path until traffic slacks off.
    (Congestion control might mitigate this to the extent that one
    is talking to just one source, but with multiple sessions, one
    will still have to keep up.) To get the buffer emptied to make
    space for further drops and to get the latency back down, one
    will have to size the backplane bandwidth wider and have the
    iSCSI engine process at a higher data rate.
    With FIM or COWS, one can end the incident with much less data
    in the buffer as one processes PDUs after the hole while waiting
    for it to be filled. Then, when the hole is filled one just has
    to process a small amount of data to catch up.
    Pat Thaler
    -----Original Message-----
    From: WENDT,JIM (HP-Roseville,ex1) []
    Sent: Tuesday, January 29, 2002 10:47 PM
    Subject: iSCSI: No Framing
    So, it would be good to hear from several iSCSI
    NIC/chip implementors who:
    - have or plan to implement FIM or COWS (or some
    other framing mechanism) and take advantage of it to
    minimize demands on on-NIC buffer memory
    - believe that all-buffers-on-chip solutions are
    feasible and valid (wrt the points above, including
    - can quantify the memory/pin/power/space cost
    savings for all-buffers-on-chip solutions


Last updated: Mon Feb 04 17:17:58 2002
8626 messages in chronological order