Re: Framing Discussion

[Mohan Parthasarathy <Mohan.Parthasarathy@eng.sun.com>]

> At 05:34 PM 12/20/2000 -0800, Mohan Parthasarathy wrote:
> >If every TCP segment has a iscsi header, which is a waste, then this
> >problem is relatively simple in identifying which iscsi command
> >a given TCP segment belongs to and also indexing into the right
> >offset inside the buffer.
> 
> Why would such a solution be a waste?  As a percentage of bandwidth, this 
> falls very much into the noise category even with 1500 Byte segments.  Most 
> high-speed protocols including new ones such as InfiniBand use a small 
> header per packet to allow hardware a simple mechanism to understand where 
> to place the data (also applies to software implementations which can do 
> better with minor hardware assists).
>
It is a waste, if there is a better solution to tackle the same thing.
If this can make things simpler, we should consider this as an option.
But how does the sender insert this header ? Typically the target
side would just dump all the data to TCP - which contains a iscsi
header followed by data. TCP then would segment the data depending
on the MSS. How do you insert such an header for each MSS sized
data ?

The only RDMA proposal on Infinaband i have seen is the one from
Microsoft. I am not sure which one you are referring to.

> At the first BOF, I spoke about aligning the protocol with InfiniBand since 
> that will eventually become the server point of attach in the coming 
> years.  The suggestion was made to include the same RDMA semantics (if they 
> are supported) as InfiniBand.  It was further suggested there an in other 
> e-mail that a simple 8-byte header with a 4-byte CRC be associated with 
> each segment and that these fields be contained within the data payload so 
> that TCP is not impacted.  The contents of this header would contain a 
> 8-bit op-code, VA, length, etc. allowing the responder to target the memory 

If VA is the virtual address, then this has security problems. How do
you prevent arbitrary packets over-writing memory ? Actually all
you need is a tag to identify the buffer pool, offset within a
stream, and some smarts in the h/w to associate the tag to a
buffer pool. Is this not sufficient ?

> on the host / controller if all fields were valid.  If there was an 
> out-of-order delivery, the data could be spilled to temporary memory either 
> in the host or the adapter and upon recovery, delivered to the correct 
> target buffer without requiring host processor intervention with a little 
> creativity.  This 12-bytes of overhead for SCSI operations would have 
> minimal impact on link utilization and overall solution efficiency.  I 
> believe these same concepts have been stated in the various RDMA proposals 
> that have been distributed and given the eventual movement to InfiniBand 
> for servers and the new SRP (SCSI RDMA Protocol), one might want to create 
> an iSCSI solution that can easily bridge into these other technologies.
> 
> Note: The arguments about adapter complexity, impact to OS, etc. are rather 
> moot in many ways.  The work will be done to support InfiniBand over the 
> next couple of years and thus the cost to implement / support is going to 
> be fairly minimal.  It should also be noted that many of these changes have 
> already be done using PCI / PCI-X based solutions that support VIA, 
> Scheduled Transfer, Oracle, MPI, Sockets Direct, etc. so the ability to 
> deploy solutions in the highly desirable I/O interconnect independent way 
> is available today as well.  One does need to wait or rely upon InfiniBand 
> to make all of this happen.  It should also be noted that many companies 
> will be working to have Linux support for this type of technology in the 
> upcoming year so solutions should be available to all by the time iSCSI 
> ramps to volume in 2002.
>
I am not sure which proposal you are talking about. Infiniband working
group is looking at the proposal from microsoft for RDMA. I think it
is more specific to Infiniband.

-mohan

> Mike
>