[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

    RE: iSCSI: No Framing

    Hi Rod,
    Your are correct for the situation where there is only one iSCSI session.
    One of the benefits of framing is that when there are multiple sessions, 
    a hole in one session won't hold up the other sessions.
    >-----Original Message-----
    >From: Rod Harrison []
    >Sent: Monday, February 04, 2002 8:36 AM
    >To: THALER,PAT (A-Roseville,ex1); WENDT,JIM (HP-Roseville,ex1);
    >Subject: RE: iSCSI: No Framing
    >	Framing of any kind is not a panacea allowing for 
    >almost buffering
    >free target implementations, it can only move the buffering from the
    >transport layer up to the iSCSI layer. Remember that the target MUST
    >guarantee commands are delivered to SCSI in the command sequence
    >number order. This means if a dropped packet contains an iSCSI command
    >PDU you are going to have to buffer until the CmdSN hole is plugged.
    >	Consider the example of an initiator filling the command window,
    >perhaps sending CmdSNs 1 through 100, possibly along with immediate or
    >unsolicited data for some of the commands. If the packet dropped
    >contained the command PDU for CmdSN 50 then the target iSCSI layer
    >must buffer CmdSNs 51 through 100 and their associated DATA_OUT PDUs.
    >Since you can continue to process any DATA_OUT PDUs for CmdSNs 1
    >through 49 the buffering required is reduced, but not eliminated,
    >	Only if the packet dropped contains DATA_OUTs and no 
    >command PDUs can
    >you continue processing past the hole while waiting for it to be
    >filled. You can R2T for data associated with commands after the hole
    >but that serves only to potentially decrease recovery time, not reduce
    >buffer requirements.
    >	Don't get me wrong, I'm not saying framing isn't a good 
    >thing but it
    >is only one piece of the puzzle.
    >	- Rod
    >-----Original Message-----
    >From: []On Behalf Of
    >THALER,PAT (A-Roseville,ex1)
    >Sent: Sunday, February 03, 2002 11:01 PM
    >To: WENDT,JIM (HP-Roseville,ex1);
    >Subject: RE: iSCSI: No Framing
    >In answer to your questions:
    >Agilent is planning to implement framing. We view both FIM and
    >COWS as having about the same utility so we would implement
    >whichever one went into the standard.
    >A smoothing buffer on the chip is feasible wrt your point 2.
    >There are some configurations that would use external memory.
    >Also, one concern is that the very situation where one would
    >need large window size for efficiency - high bandwidth long
    >distance communication - is where out of order receipt becomes
    >increasingly likely so the amount of memory for this situation
    >could push up cost to an excessive extent.
    >Reducing the amount of traffic that has to be shunted to an
    >external memory affects the bandwidth that needs to be provided
    >for that memory. If we can handle most PDUs internally then the
    >external memory doesn't have to be as wide and as fast. For
    >instance, if a drop means that the iSCSI PDU with the drop
    >is put into external memory then that takes less memory bandwidth
    >than if a drop means the whole data stream for that session will
    >be going into the buffer until the hole is filled.
    >This decision also effects latency after a drop and required
    >backplane bandwidth.
    >Let's say one is on a 10 Gbit/s extreme long distance WAN
    >connection and that there is a drop. If the round trip delay
    >for getting the whole filled is 100 ms, then without framing,
    >one could have about 1 Gbit stored in the local buffer memory
    >by the time the hole is filled. One will have to process all
    >of this through iSCSI and across the backplane before one can
    >deal with the new traffic coming in. If the backplane data rate
    >is closely sized to the external data rate, an extra 100 ms latency
    >has just been inserted into the path until traffic slacks off.
    >(Congestion control might mitigate this to the extent that one
    >is talking to just one source, but with multiple sessions, one
    >will still have to keep up.) To get the buffer emptied to make
    >space for further drops and to get the latency back down, one
    >will have to size the backplane bandwidth wider and have the
    >iSCSI engine process at a higher data rate.
    >With FIM or COWS, one can end the incident with much less data
    >in the buffer as one processes PDUs after the hole while waiting
    >for it to be filled. Then, when the hole is filled one just has
    >to process a small amount of data to catch up.
    >Pat Thaler
    >-----Original Message-----
    >From: WENDT,JIM (HP-Roseville,ex1) []
    >Sent: Tuesday, January 29, 2002 10:47 PM
    >Subject: iSCSI: No Framing
    >So, it would be good to hear from several iSCSI
    >NIC/chip implementors who:
    >- have or plan to implement FIM or COWS (or some
    >other framing mechanism) and take advantage of it to
    >minimize demands on on-NIC buffer memory
    >- believe that all-buffers-on-chip solutions are
    >feasible and valid (wrt the points above, including
    >- can quantify the memory/pin/power/space cost
    >savings for all-buffers-on-chip solutions


Last updated: Mon Feb 04 15:17:58 2002
8622 messages in chronological order