Re: Framing Discussion

[Michael Krause <krause@cup.hp.com>]

At 05:34 PM 12/20/2000 -0800, Mohan Parthasarathy wrote:
>If every TCP segment has a iscsi header, which is a waste, then this
>problem is relatively simple in identifying which iscsi command
>a given TCP segment belongs to and also indexing into the right
>offset inside the buffer.

Why would such a solution be a waste?  As a percentage of bandwidth, this 
falls very much into the noise category even with 1500 Byte segments.  Most 
high-speed protocols including new ones such as InfiniBand use a small 
header per packet to allow hardware a simple mechanism to understand where 
to place the data (also applies to software implementations which can do 
better with minor hardware assists).

At the first BOF, I spoke about aligning the protocol with InfiniBand since 
that will eventually become the server point of attach in the coming 
years.  The suggestion was made to include the same RDMA semantics (if they 
are supported) as InfiniBand.  It was further suggested there an in other 
e-mail that a simple 8-byte header with a 4-byte CRC be associated with 
each segment and that these fields be contained within the data payload so 
that TCP is not impacted.  The contents of this header would contain a 
8-bit op-code, VA, length, etc. allowing the responder to target the memory 
on the host / controller if all fields were valid.  If there was an 
out-of-order delivery, the data could be spilled to temporary memory either 
in the host or the adapter and upon recovery, delivered to the correct 
target buffer without requiring host processor intervention with a little 
creativity.  This 12-bytes of overhead for SCSI operations would have 
minimal impact on link utilization and overall solution efficiency.  I 
believe these same concepts have been stated in the various RDMA proposals 
that have been distributed and given the eventual movement to InfiniBand 
for servers and the new SRP (SCSI RDMA Protocol), one might want to create 
an iSCSI solution that can easily bridge into these other technologies.

Note: The arguments about adapter complexity, impact to OS, etc. are rather 
moot in many ways.  The work will be done to support InfiniBand over the 
next couple of years and thus the cost to implement / support is going to 
be fairly minimal.  It should also be noted that many of these changes have 
already be done using PCI / PCI-X based solutions that support VIA, 
Scheduled Transfer, Oracle, MPI, Sockets Direct, etc. so the ability to 
deploy solutions in the highly desirable I/O interconnect independent way 
is available today as well.  One does need to wait or rely upon InfiniBand 
to make all of this happen.  It should also be noted that many companies 
will be working to have Linux support for this type of technology in the 
upcoming year so solutions should be available to all by the time iSCSI 
ramps to volume in 2002.

Mike