Re: iSCSI Autosense Consensus, Connection next steps

["Randall R. Stewart" <randall@stewart.chicago.il.us>]

Ok,

explain something to me...

csapuntz@cisco.com wrote:
> 
> Asymmetric connections may be the way to go.
> 
> Let's look at characteristics of control vs. data:
> 
> * iSCSI control
>         - low bandwidth (handled on a CPU)
>         - should generally be processed in FIFO order
>         - complex state machines (probably software implemented)
>         - flow control nice
>         - doesn't need to end up in any special buffers
>           (can be processed in-place)
> 
> * iSCSI data/ready to transmit (rtt)
>         - potentially high bandwidth
>         - with appropriate headers, segments can be
>           processed entirely out-of-order
>         - simple data transfer state machine
>         - no flow control on data needed, just congestion control
>                    - rtt's need to be flow-controlled, but
>                      that can be done with a simple credit mechanism
>         - data needs to end up in special buffers (e.g. buffer cache)
>
I understand how you will obtain all of the above with 
two TCP connecions. I can even support this, considering
the 1 for control is "low bandwidth" (note: I can't support
N data connections.. only 1). 

But my question is to the Data connection. How do you get 
"no flow control on data needed, just congestion control" on
a TCP connection?


> The processing requirements of the data and control seem quite
> asymmetric. Mixing them on one TCP channel makes separating out the
> data and control more difficult.
> 
> Conversely, by putting control and data on separate TCP connections,
> you can use the flow label (IP addresses, TCP ports) to determine
> whether to route this along the control or data path in your machine.
> 
> So, I think there is value in the proposal that we use 1 control
> connection and up to n separate dedicated data/RTT connections.

No I can NOT agree with N seperate dedicated data/RTT connections. I
think you should go look at Sally's email that was posted here
earlier... here it is...
**************************************************************************
Franco Travostino wrote:
> 
> A bit of Related Work ... On the forever-young 1 vs. N TCP connections
> subject, you may be interested in the results shown in the '97 "An
> Application-Level Solution to TCP's Satellite Inefficiencies" paper
> (available at http://jarok.cs.ohiou.edu/projects/satellite/) and on Sally
> Floyd's comment hereafter. The intersection with satellite links is quite
> accidental, and the arguments/results apply above and beyond satellite links.
> 
> -franco
> 
> >From majordom@ISI.EDU Fri Dec  6 23:05:58 1996
> >To: fred@cisco.com (Fred Baker)
> >Cc: end2end-tf@isi.edu
> >Subject: Re: Related paper/re:satellites
> >Date: Fri, 06 Dec 1996 23:05:47 PST
> >From: Sally Floyd <floyd@ee.lbl.gov>
> >Sender: owner-end2end-tf@ISI.EDU
> >Precedence: bulk
> >Content-Length: 2309
> >X-Lines: 44
> >
> >Fred -
> >
> >>You might be interested in reviewing this paper, which is what I'm
> >>discussing with Karen Hansen, Dan Shell, and the folks from Comsat. It
> >>relates to some TCP/Satellite work being done at NASA Lewis Research
> >>Center.
> >
> >Basically (from a quick read), the paper, on "An Application-Level
> >Solution to TCP's Satellite Inefficiencies", recommends breaking a
> >single TCP connection into multiple connections at the
> >application level, to increase throughput on satellite circuits.
> >It is not too surprising that this increases the TCP throughput, but it
> >is still not a good idea.  For a single TCP connection, a single packet
> >drop results in the throughput for that connection being cut by half,
> >and then increased by roughly one packet per RTT.  For a TCP connection
> >that has instead been separated into N different TCP connections, a
> >single packet drop results in one of the N connections, receiving
> >1/N-th of the total bandwidth, having its throughput cut in half.  Thus
> >the bandwidth of the aggregate connection has its bandwidth reduced to
> >(1 - 1/(2N))-th of its former bandwidth - that is, the larger the value
> >for N, the smaller the aggregate bandwidth is cut.  And then, because
> >each TCP connection continues to increase its congestion window by one
> >packet per RTT, for those TCP connections that have not yet reached the
> >receiver's advertised window, the aggregate TCP connections together
> >increase their bandwidth by up to N packets per RTT.
> >
> >Summarizing, splitting a TCP connection into N separate connections
> >simply increases the aggressiveness of the TCP congestion control.
> >Meaning that this TCP is now more likely to "steal" bandwidth from
> >other TCP connections in times of congestion.  And increasing the
> >aggressiveness of the TCP congestion control too far (by choosing N too
> >large) is counterproductive even for the aggregate TCP connection, as
> >the paper shows.
> >
> >I would suggest that this is exactly the kind of development for which
> >[RED is needed].
> >
> >- Sally
> >
> >
> >
******************************************************************
In my mind it will be a hard sell to the IESG to ask for
multiple connections for high-bandwidth data? Scott/Allision any
comments on this?

> 
> The symmetric, connection allegiance case was developed when we were
> smoking some really good stuff in Haifa. It really addresses a
> corner case: minimizing the communication needs between NICs in systems
> that will stripe requests across NICs.
> 
> However, somewhere along the line, we forgot to optimize for the
> single NIC case.
> 
> ------------------
> 
> Also using a separate TCP connections may allow us to migrate one
> day to an architecture where a different node in the IP network
> returns the actual data from a SCSI READ!
> 
> Client <---- iSCSI Control ----> Storage Controller <---- TBD ----> Disk array
>   ^                                                                    ^
>   +-------------------------------iSCSI Data protocol -----------------+
> 
> -Costa

-- 
Randall R. Stewart
randall@stewart.chicago.il.us or rrs@cisco.com
815-342-5222 (cell) 815-477-2127 (work)