Re: Framing Discussion

[Mohan Parthasarathy <Mohan.Parthasarathy@eng.sun.com>]

> At 10:18 AM 12/22/2000 -0800, Mohan Parthasarathy wrote:
> 
> >It is a waste, if there is a better solution to tackle the same thing.
> >If this can make things simpler, we should consider this as an option.
> >But how does the sender insert this header ? Typically the target
> >side would just dump all the data to TCP - which contains a iscsi
> >header followed by data. TCP then would segment the data depending
> >on the MSS. How do you insert such an header for each MSS sized
> >data ?
> 
> This is a simple gather operation.  Most OS understand how to gather 
> different segments into a single byte stream and also account for 
> additional headers.  Nothing special here.
>
Sure it is possible but generally not the optimized path in the stack.
But there are other problems with this proposal i guess. As TCP
is stream oriented it need not obey boundaries of the various
segments. The effect is you won't see a iscsi header on every
datagram.
> 
> 
> > > At the first BOF, I spoke about aligning the protocol with InfiniBand 
> > since
> > > that will eventually become the server point of attach in the coming
> > > years.  The suggestion was made to include the same RDMA semantics (if 
> > they
> > > are supported) as InfiniBand.  It was further suggested there an in other
> > > e-mail that a simple 8-byte header with a 4-byte CRC be associated with
> > > each segment and that these fields be contained within the data payload so
> > > that TCP is not impacted.  The contents of this header would contain a
> > > 8-bit op-code, VA, length, etc. allowing the responder to target the 
> > memory
> >
> >If VA is the virtual address, then this has security problems. How do
> >you prevent arbitrary packets over-writing memory ? Actually all
> >you need is a tag to identify the buffer pool, offset within a
> >stream, and some smarts in the h/w to associate the tag to a
> >buffer pool. Is this not sufficient ?
> 
> A VA is an address but what that address is a local issue and is only used 
> as a key to find the real address.  As such, I don't see any security issue 
> beyond that associated with any datea exchange on a network and the 
> techniques even used today within FC products can be applied so the 
> technology impacts are actually well understood and can be implemented by 
> multiple vendors.
>

Normal packets - i.e not using RDMA, can't write anywhere. But i can see
how that can be done with RDMA i.e restict the region accessed by
the NIC for a given RDMA transaction.  Does the FC products do this ?

> > > on the host / controller if all fields were valid.  If there was an
> > > out-of-order delivery, the data could be spilled to temporary memory 
> > either
> > > in the host or the adapter and upon recovery, delivered to the correct
> > > target buffer without requiring host processor intervention with a little
> > > creativity.  This 12-bytes of overhead for SCSI operations would have
> > > minimal impact on link utilization and overall solution efficiency.  I
> > > believe these same concepts have been stated in the various RDMA proposals
> > > that have been distributed and given the eventual movement to InfiniBand
> > > for servers and the new SRP (SCSI RDMA Protocol), one might want to create
> > > an iSCSI solution that can easily bridge into these other technologies.
> > >
> > > Note: The arguments about adapter complexity, impact to OS, etc. are 
> > rather
> > > moot in many ways.  The work will be done to support InfiniBand over the
> > > next couple of years and thus the cost to implement / support is going to
> > > be fairly minimal.  It should also be noted that many of these changes 
> > have
> > > already be done using PCI / PCI-X based solutions that support VIA,
> > > Scheduled Transfer, Oracle, MPI, Sockets Direct, etc. so the ability to
> > > deploy solutions in the highly desirable I/O interconnect independent way
> > > is available today as well.  One does need to wait or rely upon InfiniBand
> > > to make all of this happen.  It should also be noted that many companies
> > > will be working to have Linux support for this type of technology in the
> > > upcoming year so solutions should be available to all by the time iSCSI
> > > ramps to volume in 2002.
> > >
> >I am not sure which proposal you are talking about. Infiniband working
> >group is looking at the proposal from microsoft for RDMA. I think it
> >is more specific to Infiniband.
> 
> I believe you are referring to the InfiniBand application workgroup which 
> has been looking at how to bootstrap InfiniBand via SRP (SCSI RDMA 
> Protocol) which is one possible mechanism to support storage over an 
> InfiniBand fabric - there are several mechanisms being defined for 

This is Winsock direct proposal. It came to the IBTA working group
i think. I have the document with me. This proposes to bypass the
normal tcp/ip stack and use RDMA to achieve high throughput.

-mohan

> boot.  This is a T10 not a Microsoft effort.  InfiniBand is simply trying 
> to accelerate some of this work but T10 will own and drive it to 
> completion. There is an upcoming T10 meeting Houston in January focused on 
> SRP that people might want to attend concerning InfiniBand.
> 
> Mike
>