RE: iSCSI: Immediate Data

[Robert Snively <rsnively@Brocade.COM>]

John,

Some minor disagreements here:

> 1. Immediate Data permits the elimination of an additional network
> handshake/interaction.

Immediate Data is not necessary to eliminate the handshake.  Use
Unsolicited Data with InitialR2T = no, and (assuming you have a
decent initiator) you get the same result.

> 
> 2. This is important since some applications have a key sensitivity to
> Latency. The most important of these is Database.

Database is certainly sensitive to latency, but the real
latency is to the SCSI Response, which formalizes the promise
that the data will never, ever, be lost from the storage.
Even the use of InitialR2T = yes on long links is a second-order
performance effect on this if your storage is a disk drive.

> 3. Not only is Database sensitive to Latency, it generally 
> has small I/O
> writes.

This is application dependent.  Most database programs out of the
box tend to use small block transfers, but well-tuned databases
that are doing work other than OLTP tend to cluster writes into
much larger blocks when possible.

> 4. The hardware HBAs that many of you folks are building, 
> include a TCP/IP
> Offload Engines (TOEs).  This permits support of Gigabit Line 
> speed without
> Server Load.  However, even though many of you are attempting to
> parallelize as much of the TOE processing as possible, TCP/IP 
> processing
> will still add latency to each iSCSI interaction, as compared to Fibre
> Channel.

What can I say?  You have hit a basic truth.

....  snip   ....

> 
> 7. So it is the Immediate Data feature of iSCSI that will 
> make an important
> difference in the Key Response Time Sensitive Applications, 
> which by luck
> only transfer a small amounts of data at a time.

See 1-3.

> 8. When we attempt to let iSCSI shine on the "at-distance" storage
> environment, the value of Immediate Data, and Unsolicited 
> Data are even
> more valuable.  However, my concern at this time is for 
> Immediate Data.

I am absolutely in agreement with your assessment of unsolicited
data at distance, but immediate data is not a necessary 
part of that improvement.

As for simplicity:

> A. Creating a Buffer Manager that can allocate buffers that 
> are chained
> together, is kind of fundamental to the high performance 
> environment we are
> attempting to work with.  All the processes (whether iSCSI or 
> SCSI) will
> allocate buffers (from the Buffer Manager), place data in one 
> or more of
> these buffers, and pass the pointer list to the next process.  
> There will be
> no copying of data.

See my previous note considering the actual destination of
data, which is not in a memory address space.  It is in this
way that storage differs from networking.  You can't pass
pointers about, but must place the data into a storage
space that has the properties of non-volatility, redundancy,
coherence, and accessibility through the SCSI command set.

...   snip   ...