RE: iSCSI: No Framing

["Joe Gervais" <joeg@alacritech.com>]

It seems that markers/framing is an issue open for debate for some time to
come. In my mind that indicates that this is still experimental in nature
and should be moved to an experimental RFC, outside of the scope of the
iSCSI RFC. It seems that this is trying to optimize the case of a high
bandwidth, long haul line. Certainly, iSCSI enables that, but also enables a
much broader market for IP Storage. As Jim Wendt originally pointed out at
the start of the thread, there are solutions today. Maybe these solutions
are not as elegant as one would like to architect, but they can be made to
work with Gigabit Ethernet and 10 Gigabit Ethernet.

I am in favor of moving framing out to an experimental draft, and finishing
this standard. Contrary to his opinion, Jim Pinkerton did not hear all
TOE/NIC vendors in favor of FIM.

Joe Gervais
Alacritech

-----Original Message-----
From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
Somesh Gupta
John,

I have to point for the record, that I was the one
that proposed markers over a year ago (not that many
other did not also have this idea). I have also tried
to keep this debate at a technical level. There
are reasonable folks on both sides of the debate,
and we should just continue in that spirit.

Somesh

> > > -----Original Message-----
> > > From: WENDT,JIM (HP-Roseville,ex1) [mailto:jim_wendt@hp.com]
> > > Sent: Tuesday, January 29, 2002 10:47 PM
> > > To: ips@ece.cmu.edu
> > > Subject: iSCSI: No Framing
> > >
> > >
> > > Perhaps we should discuss the possibility of not
> > > specifying any framing mechanism (FIM or COWS) in the
> > > first version of iSCSI.
> > >
> > > The arguments for not specifying framing include
> > > (there may be others as well):
> > > 1) The brute force approach of putting TCP receiver
> > > reassembly memory on the iSCSI NIC will work for both
> > > 1Gbps and 10Gbps. Cost is incurred for memory chips,
> > > ASIC pins, power, and board space. But, it is a
> > > feasible approach.
> > >
> > > 2) I/O bus latency (or bandwidth limitations) could
> > > mandate inbound buffering (as a 'smoothing' buffer)
> > > from the iSCSI NIC to the host system. If this buffer
> > > memory is large enough to have to be off-chip, then
> > > it will require some minimum number of memory chips
> > > to provide the necessary bandwidth, and the
> > > memory/pins/power/space costs will be incurred
> > > anyway.
> > >
> > > 3) Very large receive buffering on the NIC is only
> > > required for high-bandwidth links traversing large
> > > geographic distances (which may not be the
> > > predominate case). These specialized implementations
> > > can cost more (e.g. more memory/power/pins/etc) and
> > > customers will likely be willing to pay accordingly.
> > >
> > > 4) Many NICs will likely aspire to be combo
> > > iSCSI+TOE+Ethernet NICs allowing the host system to
> > > use a single Ethernet port for all of its network
> > > communications. The TOE function on this combo NIC
> > > will already require TCP receive buffering and off-
> > > chip buffer memory to support the existing sockets
> > > interface receive model.  More importantly, to allow
> > > senders to assume ownership of the buffer upon return
> > > from a socket send call, the TOE NIC would need to
> > > copy application send buffers into NIC memory as
> > > well.
> > >
> > > 5) The framing/marker mechanism will be an iSCSI-only
> > > solution. It is quite possible that the problem will
> > > be solved via a different, and hopefully more
> > > general, mechanism for other ULPs.
> > >
> > > 6) Both FIM and COWS are poor solutions for software
> > > implementation. COWS requires, at a minimum, the
> > > sender to read every word in the buffer. FIM either
> > > imposes additional sender gather processing (iovecs)
> > > or requires a data buffer copy (on systems that don't
> > > support HW gather on sends).
> > >
> > > 7) Unless all senders thoroughly test framing/markers
> > > now (i.e. unless the framing method is a MUST
> > > implement), there is potential for future
> > > interoperability problems as framing/marker receivers
> > > are deployed in the future.
> > >
> > > 8) Neither FIM nor COWS is an elegant solution. Are
> > > we comfortable with the legacy we are creating under
> > > the umbrella of state-of-the-art IP networked
> > > storage?
> > >
> > > 9) Is it essential to build in forward compatibility
> > > now, or will there be a different solution in the
> > > 10Gbps or 40Gbps timeframe - perhaps involving iSCSI-
> > > 2?  Will it be reasonable to update or bridge initial
> > > 1Gbps deployments?
> > >
> > >
> > > So, it would be good to hear from several iSCSI
> > > NIC/chip implementors who:
> > > - have or plan to implement FIM or COWS (or some
> > > other framing mechanism) and take advantage of it to
> > > minimize demands on on-NIC buffer memory
> > > bandwidth/quantity.
> > > - believe that all-buffers-on-chip solutions are
> > > feasible and valid (wrt the points above, including
> > > #2)
> > > - can quantify the memory/pin/power/space cost
> > > savings for all-buffers-on-chip solutions
> > >
> > > Jim
> > >
> >
> >
> >
> >
>
>
>
>