[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

    RE: iSCSI: No Framing

    > Perhaps we should discuss the possibility of not 
    > specifying any framing mechanism (FIM or COWS) in the 
    > first version of iSCSI.
    I agree with all your comments, and this seems to be the
    reasonable approach to take. Brute force method, as you 
    pointed out correctly, may be acceptable for NICs/Off-load engines
    working over long-fat pipes.
    Aarohi Communications, Inc. 
    > The arguments for not specifying framing include 
    > (there may be others as well):
    > 1) The brute force approach of putting TCP receiver 
    > reassembly memory on the iSCSI NIC will work for both 
    > 1Gbps and 10Gbps. Cost is incurred for memory chips, 
    > ASIC pins, power, and board space. But, it is a 
    > feasible approach.
    > 2) I/O bus latency (or bandwidth limitations) could 
    > mandate inbound buffering (as a 'smoothing' buffer) 
    > from the iSCSI NIC to the host system. If this buffer 
    > memory is large enough to have to be off-chip, then 
    > it will require some minimum number of memory chips 
    > to provide the necessary bandwidth, and the 
    > memory/pins/power/space costs will be incurred 
    > anyway.
    > 3) Very large receive buffering on the NIC is only 
    > required for high-bandwidth links traversing large 
    > geographic distances (which may not be the 
    > predominate case). These specialized implementations 
    > can cost more (e.g. more memory/power/pins/etc) and 
    > customers will likely be willing to pay accordingly.
    > 4) Many NICs will likely aspire to be combo 
    > iSCSI+TOE+Ethernet NICs allowing the host system to 
    > use a single Ethernet port for all of its network 
    > communications. The TOE function on this combo NIC 
    > will already require TCP receive buffering and off-
    > chip buffer memory to support the existing sockets 
    > interface receive model.  More importantly, to allow 
    > senders to assume ownership of the buffer upon return 
    > from a socket send call, the TOE NIC would need to 
    > copy application send buffers into NIC memory as 
    > well.
    > 5) The framing/marker mechanism will be an iSCSI-only 
    > solution. It is quite possible that the problem will 
    > be solved via a different, and hopefully more 
    > general, mechanism for other ULPs.
    > 6) Both FIM and COWS are poor solutions for software 
    > implementation. COWS requires, at a minimum, the 
    > sender to read every word in the buffer. FIM either 
    > imposes additional sender gather processing (iovecs) 
    > or requires a data buffer copy (on systems that don't 
    > support HW gather on sends).
    > 7) Unless all senders thoroughly test framing/markers 
    > now (i.e. unless the framing method is a MUST 
    > implement), there is potential for future 
    > interoperability problems as framing/marker receivers 
    > are deployed in the future.
    > 8) Neither FIM nor COWS is an elegant solution. Are 
    > we comfortable with the legacy we are creating under 
    > the umbrella of state-of-the-art IP networked 
    > storage?
    > 9) Is it essential to build in forward compatibility 
    > now, or will there be a different solution in the 
    > 10Gbps or 40Gbps timeframe - perhaps involving iSCSI-
    > 2?  Will it be reasonable to update or bridge initial 
    > 1Gbps deployments?
    > So, it would be good to hear from several iSCSI 
    > NIC/chip implementors who:
    > - have or plan to implement FIM or COWS (or some 
    > other framing mechanism) and take advantage of it to 
    > minimize demands on on-NIC buffer memory 
    > bandwidth/quantity.
    > - believe that all-buffers-on-chip solutions are 
    > feasible and valid (wrt the points above, including 
    > #2)
    > - can quantify the memory/pin/power/space cost 
    > savings for all-buffers-on-chip solutions
    > Jim

    • References:


Last updated: Wed Jan 30 13:17:53 2002
8565 messages in chronological order