Re: iSCSI Autosense Consensus, Connection next steps

["Randall R. Stewart" <randall@stewart.chicago.il.us>]

I would hope one would use multiple NIC's... multi-homing
is a nice feature...

What I wonder about is if you have the multiple connections
up through the same path to work around TCP CC (as talked about
in Sally's email). As long as you can restrict things so
that only one connection per NIC is up between target and
host.. I don't think I have a problem with that.. unless of
course they are all on the same sub-net.. but of course this
defeats the purpose of the redundancy.. unless of course the
redundancy is there to get a fatter pipe by working around 
the CC algorithms and increasing your low level NIC card buffers..

Having support in the transport for the multi-homing is IMHO a
better solution since that way you CAN fail over to an alternate
on the first retransmission....

R

        

John Hufferd/San Jose/IBM wrote:
> 
> I think there is some serious miss reading of the paper, or perhaps miss
> understanding on what we are trying to do with iSCSI.  The paper talks
> about breaking a TCP/IP connection into multiple paths (connections).
> There is no iSCSI proposal to do this.  In all cases, the iSCSI proposals
> talk about the data for a command returning on ONE connection (which
> connection it is maybe variable).
> 
> To say that we should not use multiple NICs for iSCSI is perhaps not
> reasonable.
> 
> .
> .
> .
> John L. Hufferd
> 
> "Randall R. Stewart" <randall@stewart.chicago.il.us>@ece.cmu.edu on
> 09/06/2000 06:36:38 AM
> 
> Sent by:  owner-ips@ece.cmu.edu
> 
> To:   csapuntz@cisco.com
> cc:   ips@ece.cmu.edu, Allison Mankin <mankin@east.isi.edu>, Scott Bradner
>       <sob@harvard.edu>
> Subject:  Re: iSCSI Autosense Consensus, Connection next steps
> 
> Ok,
> 
> explain something to me...
> 
> csapuntz@cisco.com wrote:
> >
> > Asymmetric connections may be the way to go.
> >
> > Let's look at characteristics of control vs. data:
> >
> > * iSCSI control
> >         - low bandwidth (handled on a CPU)
> >         - should generally be processed in FIFO order
> >         - complex state machines (probably software implemented)
> >         - flow control nice
> >         - doesn't need to end up in any special buffers
> >           (can be processed in-place)
> >
> > * iSCSI data/ready to transmit (rtt)
> >         - potentially high bandwidth
> >         - with appropriate headers, segments can be
> >           processed entirely out-of-order
> >         - simple data transfer state machine
> >         - no flow control on data needed, just congestion control
> >                    - rtt's need to be flow-controlled, but
> >                      that can be done with a simple credit mechanism
> >         - data needs to end up in special buffers (e.g. buffer cache)
> >
> I understand how you will obtain all of the above with
> two TCP connecions. I can even support this, considering
> the 1 for control is "low bandwidth" (note: I can't support
> N data connections.. only 1).
> 
> But my question is to the Data connection. How do you get
> "no flow control on data needed, just congestion control" on
> a TCP connection?
> 
> > The processing requirements of the data and control seem quite
> > asymmetric. Mixing them on one TCP channel makes separating out the
> > data and control more difficult.
> >
> > Conversely, by putting control and data on separate TCP connections,
> > you can use the flow label (IP addresses, TCP ports) to determine
> > whether to route this along the control or data path in your machine.
> >
> > So, I think there is value in the proposal that we use 1 control
> > connection and up to n separate dedicated data/RTT connections.
> 
> No I can NOT agree with N seperate dedicated data/RTT connections. I
> think you should go look at Sally's email that was posted here
> earlier... here it is...
> **************************************************************************
> Franco Travostino wrote:
> >
> > A bit of Related Work ... On the forever-young 1 vs. N TCP connections
> > subject, you may be interested in the results shown in the '97 "An
> > Application-Level Solution to TCP's Satellite Inefficiencies" paper
> > (available at http://jarok.cs.ohiou.edu/projects/satellite/) and on Sally
> > Floyd's comment hereafter. The intersection with satellite links is quite
> > accidental, and the arguments/results apply above and beyond satellite
> links.
> >
> > -franco
> >
> > >From majordom@ISI.EDU Fri Dec  6 23:05:58 1996
> > >To: fred@cisco.com (Fred Baker)
> > >Cc: end2end-tf@isi.edu
> > >Subject: Re: Related paper/re:satellites
> > >Date: Fri, 06 Dec 1996 23:05:47 PST
> > >From: Sally Floyd <floyd@ee.lbl.gov>
> > >Sender: owner-end2end-tf@ISI.EDU
> > >Precedence: bulk
> > >Content-Length: 2309
> > >X-Lines: 44
> > >
> > >Fred -
> > >
> > >>You might be interested in reviewing this paper, which is what I'm
> > >>discussing with Karen Hansen, Dan Shell, and the folks from Comsat. It
> > >>relates to some TCP/Satellite work being done at NASA Lewis Research
> > >>Center.
> > >
> > >Basically (from a quick read), the paper, on "An Application-Level
> > >Solution to TCP's Satellite Inefficiencies", recommends breaking a
> > >single TCP connection into multiple connections at the
> > >application level, to increase throughput on satellite circuits.
> > >It is not too surprising that this increases the TCP throughput, but it
> > >is still not a good idea.  For a single TCP connection, a single packet
> > >drop results in the throughput for that connection being cut by half,
> > >and then increased by roughly one packet per RTT.  For a TCP connection
> > >that has instead been separated into N different TCP connections, a
> > >single packet drop results in one of the N connections, receiving
> > >1/N-th of the total bandwidth, having its throughput cut in half.  Thus
> > >the bandwidth of the aggregate connection has its bandwidth reduced to
> > >(1 - 1/(2N))-th of its former bandwidth - that is, the larger the value
> > >for N, the smaller the aggregate bandwidth is cut.  And then, because
> > >each TCP connection continues to increase its congestion window by one
> > >packet per RTT, for those TCP connections that have not yet reached the
> > >receiver's advertised window, the aggregate TCP connections together
> > >increase their bandwidth by up to N packets per RTT.
> > >
> > >Summarizing, splitting a TCP connection into N separate connections
> > >simply increases the aggressiveness of the TCP congestion control.
> > >Meaning that this TCP is now more likely to "steal" bandwidth from
> > >other TCP connections in times of congestion.  And increasing the
> > >aggressiveness of the TCP congestion control too far (by choosing N too
> > >large) is counterproductive even for the aggregate TCP connection, as
> > >the paper shows.
> > >
> > >I would suggest that this is exactly the kind of development for which
> > >[RED is needed].
> > >
> > >- Sally
> > >
> > >
> > >
> ******************************************************************
> In my mind it will be a hard sell to the IESG to ask for
> multiple connections for high-bandwidth data? Scott/Allision any
> comments on this?
> 
> >
> > The symmetric, connection allegiance case was developed when we were
> > smoking some really good stuff in Haifa. It really addresses a
> > corner case: minimizing the communication needs between NICs in systems
> > that will stripe requests across NICs.
> >
> > However, somewhere along the line, we forgot to optimize for the
> > single NIC case.
> >
> > ------------------
> >
> > Also using a separate TCP connections may allow us to migrate one
> > day to an architecture where a different node in the IP network
> > returns the actual data from a SCSI READ!
> >
> > Client <---- iSCSI Control ----> Storage Controller <---- TBD ----> Disk
> array
> >   ^                                                                    ^
> >   +-------------------------------iSCSI Data protocol -----------------+
> >
> > -Costa
> 
> --
> Randall R. Stewart
> randall@stewart.chicago.il.us or rrs@cisco.com
> 815-342-5222 (cell) 815-477-2127 (work)

-- 
Randall R. Stewart
randall@stewart.chicago.il.us or rrs@cisco.com
815-342-5222 (cell) 815-477-2127 (work)