Re: iSCSI Autosense Consensus, Connection next steps

["Randall R. Stewart" <randall@stewart.chicago.il.us>]

Stephen Bailey wrote:
> 
> > About 1 year ago we had this "transport bundling" aka load sharing IN
> > SCTP. At the request of the IESG it was removed because (if I remember
> > right) it causes all sorts of problems with Congestion Control and
> > is still deemed a research topic by the IETF (belonging to the
> > IRTF).
> 
> Interesting.  This speaks volumes.
> 
> I think more than anything else this goes to the heart of the `it
> won't work' claim.  I have claimed that connection bundling support
> should either:
>   a) be in the transport (or a low network layer), because layers
>      above the transport (ULPs) do not have effective control over
>      network path.
> or
>   b) be in the highest possible layer, because the low level
>      components can not be engineered to provide the desired behavior
>      without compromise, so ultimately the user must be exposed to,
>      aware of, and control the nature of the compromises.
> 
> > It is aong the same lines (to a small degree) as to why you don't
> > want multiple TCP connections.
> 
> Very roughly.  The specific problem is that NONE of the layers in an

Yes... very very roughly :)

> IP end-point have complete control over path selection.  The IP layer
> has the best control over this, and it only controls the initial
> interface selection.  There's no robust way to know that outside of
> the first (and possibly last) hop, you're actually using separate
> paths.  If there is a shared path component, bundled connections
> through different interfaces will mess with congestion control in the
> same way as two connections between the same end-points.
> 
> This doesn't mean that it's impossible to control path selection in
> ALL cases.  It's just that you have to carefully control the network
> configuration in addition to the software behavior.
> 
> This is an issue even in the fault tolerance application.  You can not
> say ``all you need to do to get fully redundant paths is hook two NIC
> cards to each endpoint.''  If the intervening network is beyond your
> control, you have no idea, and no way to tell what multipathing you're
> actually getting.
> 

Source AND destination based routing are still a research issue. This
is what you must have to control the paths and it is NOT something
we do in the IETF .. at least not yet :)

So for now, if you really want to have a "fault tolerant" network, you
must as you say control very carefully the setup of the routers and
the NIC cards inbetween.. making sure that they are seperate and
diverse..

This has been true in the telephone world for ages though... I can
still remember an outage I was told about where they had two
seperate power feeds to a phone company building.. but both
transformers hung on the same pole... and of course a car hit
that pole :-0 luckly batteries and generators were able to 
kick in :-)


> The question is whether there should be a feature in iSCSI which
> breaks if and when lower layer path selection logic changes.  I am
> arguing that a clean separation of concerns between iSCSI and the
> lower layers on path selection will make iSCSI a more durable
> protocol.


I do like a clean seperation of layers for this. The concept is
very nice.. 

> 
> > But the actual implemenation of the load balancing is above the
> > transport layer... :0
> 
> This has to be because IETF doesn't have an acceptable solution, so
> they're punting.  They say that connection bundling messes with
> congestion control, and then the proceed to hand the responsibility
> over to layers that are completely ignorant of congestion.
> 
> Pushing this function up above the transport is not likely to
> meaningfully change the nature of the traffic flow at all.  It will
> ensure that there are numerically fewer ULPs actually using it, but
> that's about it.  A ULP solution still has the same potential failure
> modes as that same ULP using a transport level solution.

Ah, but if you think about it as to having to have been configured to
make it work reasonably then allowing the ULP the hooks and knobs to
do the load balancing is not so bad, tricky but not so bad...

> 
> Nonetheless, it certainly has been argued here that iSCSI is a ULP
> that needs this function.  If people really think that the reality
> that the feature may or may not work on a configuration by
> configuration basis is OK, and an iSCSI-specific solution offers
> something important anything over the wedge driver solution, hey, rock
> on.
> 
Hmm, I don't know about rock.. but with the right hooks I think it
may be possible to make it work. In any event it should probably be
made so that the load-balancing function is defaulted to off (if
it is added) and that it SHOULD be turned on when you have done
the careful network configuration thing :-)

R


-- 
Randall R. Stewart
randall@stewart.chicago.il.us or rrs@cisco.com
815-342-5222 (cell) 815-477-2127 (work)