Re: TCP Framing (considered helpful?)

[Michael Krause <krause@cup.hp.com>]

At 02:54 PM 5/23/2001 -0700, Mallikarjun C. wrote:
>John,
>
>Two comments.  I may have misunderstood what you're saying since
>your initial comments imply framing=warp/RDMA/messageboundary,
>and towards the end, you seem to imply framing=markers.
>
>- It's true that implementing iSCSI-level markers does not require
>   modifying TCP/IP stacks.  But using the markers (I mean to use it
>   to effectively steer) requires TCP/IP changes!  It is a separate
>   matter that it is offloaded onto the NIC.
>
>- You seem to implicitly suggest that WARP (when it becomes available)
>   must go into host software stacks to be useful, and hence cannot be
>   done in the right timeframe for iSCSI.

RDMA was never intended to be transparent to the world.  One is explicitly 
exporting memory to remote endnodes to target for specific operations - in 
many ways, not really any different than what one does with a SCSI PDU 
header.  What will need to occur for iSCSI to use RDMA is at a minimum to 
define how one translates a SCSI operation into an RDMA READ or RDMA WRITE 
operation akin to the SRP (T10 SCSI RDMA Protocol).  In this case, one 
would have all of the iSCSI session management, security, login, error 
management, etc. as defined today but the main data movement would most 
likely be a simplified SRP main data movement algorithm / mapping (in 
essence a blend of the two technologies at least on the main paths).  This 
would also allow iSCSI to bridge more easily into the InfiniBand-based 
server environment (will grow in volume over the next few years) by 
allowing a simplified bridging between the two technology / management 
domains while maintaining an end-to-end iSCSI software (kernel driver as 
well) solution.

>  Since one could envision
>   offloading WARP onto the NIC as well, I assume you're hinting

While there is clearly a software component in terms of memory management 
and communication between endnodes, the actual data movement is implemented 
in hardware.  It really isn't practical to implement this in a strictly 
software stack given the performance and solution resource impacts.  Note: 
There is the issue of maintaining virtual-to-physical mappings on the NIC 
associated with RDMA which could simply be at a minimal a mapping of all 
physical memory (preferably 2x) thereby allowing one to perform RDMA for 
any service to any address used by kernel or user-space I/O operations.

>         a) either at interoperability issues between software
>            and hardware iSCSI implementations relying on WARP,
>         b) or at interoperability issues between hardware and
>            hardware implementations.
>
>   iSCSI currently mandates a "no sync and steering" mode which
>   ensures interoperability in either case.  Perhaps you were really
>   concerned about performance in case (b) then?

It means that one has to decide during login as to whether the session 
should be established or not as a function of the attributes on either side 
of the connection. For example, a RDMA / TOE implementation may simply 
refuse to accept a login from a non-RDMA implementation as it may not have 
the resource (NIC or system) to handle a more resource intensive / lower 
performance remote endnode (might also revert to a software-based solution 
if desired which would not use really any NIC services depending upon what 
protocol stack off-load is provided).

Mike