RE: ISCSI: Urgent Flag requirement violates TCP.

["Dante Malagrino'" <dantem@cisco.com>]

At 09:18 AM 11/14/2000 -0800, Y P Cheng wrote:
>Well, the reasons for continuing to move data as fast as the wire speed in
>the face of a sequence hole are: 1) out-of-order reception is considered
>normal and happens often, 2) On a ten gigabit backbone, several milliseconds
>of delay requires the buffering of several megabytes of data on an adapter.
>(One megabyte per millisecond).  Incoming data are not limited from a single
>source.  Many nodes may send and return data to an adapter at the same time.
>At a gigabyte per second incoming rate, using several megabytes of SRAM for
>buffering on an adapter is very expensive.

1) what do you mean by "out-of-order reception is considered normal and 
happens often"? Last figures I saw talked about 3% of out-of-order TCP 
segments on WANs (worst-case). Is it a critical value?

2) with 64MB of RAM, and not supporting Window Scale Option, given a 
maximum Window Size of 64k (I have seen it very rarely), we end up having 
1k-conns concurrently communicating with a given adapter. With a more 
reasonable Window Size (16k), the number of connections is four times more. 
Do we expect to see more connections than that? Do we plan to support 
Window Scale Option?

>If there is any possibility for a TOE adapter to learn the beginning of an
>iSCSI PDU in face of a sequence hole, while it could try to keep in-order
>delivery of command and status PDUs -- although may not be necessary, the
>data PDUs can be moved quickly to the buffers pre-allocated by application
>software.  Hence, it will greatly reduce the buffering requirement of the
>adapter.

TCP is bytestream oriented, and each packet carries a Sequence Number. Why 
can't you save an out-of-order packet in his appropriate location (in the 
application buffer), and then put the missing packet in the appropriate 
hole? There is no need for copy.

>If a TOE adapter can't move at the speed of the wire, how are we going to
>take advantage of the 10 Gbps media?  On a large WAN with high speed
>backbone connections with 100 millisecond latency, there could be 100
>megabytes of data inflight.  Can we buffer all 100 megabytes on an adapter
>or should we limit the inflight data by set a small TCP window limited by
>the buffer size of the adapter?

Even with large window sizes (64k), how do we have 100MB inflight? See my 
previous comment.

>The right design of a TOE adapter is always
>to move data quickly to the buffers already allocated by application
>software and to allow as much data inflight as possible.  To achieve that,
>the TOE adapter needs all the help it can get.  If we can't move data at the
>wire speed, lets not bother to build 10 Gbps networks.

Isn't the help provided by the TCP sequence number enough?

-- Dante




Dante Malagrino'
Cisco Systems                  Empowering the Internet Generation
170 West Tasman Drive                         Tel. (408) 525 4120
San Jose, CA, 95134-1706                      Fax. (408) 525 4120
                                                  dantem@cisco.com