FW: ipfc discussion of draft-otis-fc-sctp-ip-00.txt

[Black_David@emc.com]

For those not on the ipfc list, here is the discussion
of draft-otis-fc-sctp-ip.txt on the use of SCTP and
draft-xie-stewart-usctp-00.txt on a proposed
unreliable delivery mechanism for SCTP.

Anyone replying to this message who quotes or forwards
its entire contents will be summarily shot :-).

--David

-----------------------------------------------------


From:	Randall R. Stewart [randall@stewart.chicago.il.us]
Sent:	Thursday, August 24, 2000 9:47 AM
To:	ipfc@standards.gadzoox.com
Subject:	Re:draft-otis-fc-sctp-ip-00.txt

Greetings:

(I apologize for the earlier subscribe mis-send... I should
 know better than to do anything before a second cup of
 coffee in the A.M. :->)

I have just finished reading both the above mentioned draft
and the draft - draft-ietf-ipfc-fcoverip-02.txt

I think that Douglas did a nice job of summarizing some of
the benefits that FC over SCTP/IP can gain in this draft
i.e.:

      - Headers contained within one frame. 
      - Objects aligned at 32-bit boundaries. 
      - Out of sequence frame processing. 
      - Standard authentication. 
      - Independent streams under common control. 
      - Session restart. 
      - Improved error detection. 
      - Prevention of blind spoofing and denial of service attacks. 
      - Standard Heartbeat and multi-homing. (optional) 

One other things not mentioned was:

      - Congestion control


Now after reviewing the ipfc-fcoverip draft particularly the
section that specifies why TCP should not be used, there may
be objections to the Otis draft...

One of problems mentioned in ipfc-fcoverip is the retransmission
by TCP being in conflict to retransmissions that would occur
at the FC level. I would like to call your attention to a
draft that I think will solve this problem... 

http://search.ietf.org/internet-drafts/draft-xie-stewart-usctp-00.txt

This draft defines a new extension that we have proposed for SCTP
that allows a particular stream to be specified has "no retransmission".
This way you can mix both completly reliable and unreliable transmissions
over the same association AND still maintain correct congestion control
policy's in the network. We still are debating the addition of
a couple of things to this draft:

   - Should we have the ability to turn off checksums on selected
     data portions traveling on a stream. This way one could disable
     the alder-32 checksum on a unreliable stream packet but not
     the header, control,  or any reliable data.

   - How do you handle larger than the P-MTU sized messages? Currently
     the draft dis-allows this since when sending to an unreliable stream
     you can never re-assemble the larger packet if a piece of one
     is lost. We may want to think about adding the ability to still
     fragment and hook in a mechanism to the reliable re-assembly queues
     to discard in-complete reassembly's where some datagrams are
     dropped due to the unreliable transmission. This would be desireable
     to keep from having to do a double P-MTU discover (one at the 
     SCTP layer and the other at the IPFC layer).

We would appreciate any comments on the draft and I would be more
than glad to present either here on the mailing list or in San Diego
a short overview of both SCTP or SCTP-U if this is helpful... 

Thanks for your time and I would welcome any comments or questions..

R
-- 
Randall R. Stewart
randall@stewart.chicago.il.us or rrs@cisco.com
815-342-5222

From:	Wayland Jeong [wayland@troikanetworks.com]
Sent:	Thursday, August 24, 2000 12:42 PM
To:	'Randall R. Stewart'; ipfc@standards.gadzoox.com
Subject:	RE: draft-otis-fc-sctp-ip-00.txt


[ stuff deleted ]

> Now after reviewing the ipfc-fcoverip draft particularly the
> section that specifies why TCP should not be used, there may
> be objections to the Otis draft...
> 
> One of problems mentioned in ipfc-fcoverip is the retransmission
> by TCP being in conflict to retransmissions that would occur
> at the FC level. I would like to call your attention to a
> draft that I think will solve this problem... 
> 
> http://search.ietf.org/internet-drafts/draft-xie-stewart-usctp-00.txt
> 
> This draft defines a new extension that we have proposed for SCTP
> that allows a particular stream to be specified has "no retransmission".
> This way you can mix both completly reliable and unreliable transmissions
> over the same association AND still maintain correct congestion control
> policy's in the network. We still are debating the addition of
> a couple of things to this draft:
>

The problem with this is that you don't want to rely on an I/O level retry
to solve congestion problems in the network. If frames are being dropped due
to congestion issues in the network (i.e. WRED) then having SCSI suffer
a timeout and retry an entire I/O will help exacerbate congestion issues
and bring performance to its knees. Depending on the traffic type, an I/O
level retry may result in a 64K block retransmission after a timeout which
is on the order of seconds. Unless you can absolutely rely on diffserv
and/or RSVP or some other QoS mechanism to guarantee reliable delivery,
the overall SCSI performance will suffer.

I guess you can solve the problem with a well-tuned private network or a
bandwidth matched WAN link, but this can be expensive and takes away
from the business case for this technology.

I think that any protocol which cannot tolerate lost data must have a
reliable
transport underneath it in order to achieve performance over gigabit
networks.
I'm not all that familiar with SCTP, but I thought that one of the features
that would be useful is the capability of using SACK and message numbers
to implement selective retransmission. It seems like, for storage, if we use
SCTP as the transport, we want retransmission.

-Wayland


From:	Randall R. Stewart [randall@stewart.chicago.il.us]
Sent:	Thursday, August 24, 2000 1:17 PM
To:	Wayland Jeong
Cc:	ipfc@standards.gadzoox.com
Subject:	Re: draft-otis-fc-sctp-ip-00.txt

Wayland Jeong wrote:
> 
> [ stuff deleted ]
> 
> > Now after reviewing the ipfc-fcoverip draft particularly the
> > section that specifies why TCP should not be used, there may
> > be objections to the Otis draft...
> >
> > One of problems mentioned in ipfc-fcoverip is the retransmission
> > by TCP being in conflict to retransmissions that would occur
> > at the FC level. I would like to call your attention to a
> > draft that I think will solve this problem...
> >
> > http://search.ietf.org/internet-drafts/draft-xie-stewart-usctp-00.txt
> >
> > This draft defines a new extension that we have proposed for SCTP
> > that allows a particular stream to be specified has "no retransmission".
> > This way you can mix both completly reliable and unreliable
transmissions
> > over the same association AND still maintain correct congestion control
> > policy's in the network. We still are debating the addition of
> > a couple of things to this draft:
> >
> 
> The problem with this is that you don't want to rely on an I/O level retry
> to solve congestion problems in the network. If frames are being dropped
due
> to congestion issues in the network (i.e. WRED) then having SCSI suffer
> a timeout and retry an entire I/O will help exacerbate congestion issues
> and bring performance to its knees. Depending on the traffic type, an I/O
> level retry may result in a 64K block retransmission after a timeout which
> is on the order of seconds. Unless you can absolutely rely on diffserv
> and/or RSVP or some other QoS mechanism to guarantee reliable delivery,
> the overall SCSI performance will suffer.
> 
The point of the u-sctp draft is that you DO NOT retransmit the
data. You let the upper layer do it (if it so desires). Instead
you drop your cwnd (as normal) and move forward the cumulative
TSN point. There is NO retransmission on a u-sctp stream. Now 
we also did not specify in this draft if the upper layer should
be informed of data not making it to the peer.. this may be
a good thing to add ...

> I guess you can solve the problem with a well-tuned private network or a
> bandwidth matched WAN link, but this can be expensive and takes away
> from the business case for this technology.
> 
> I think that any protocol which cannot tolerate lost data must have a
reliable
> transport underneath it in order to achieve performance over gigabit
networks.
> I'm not all that familiar with SCTP, but I thought that one of the
features
> that would be useful is the capability of using SACK and message numbers
> to implement selective retransmission. It seems like, for storage, if we
use
> SCTP as the transport, we want retransmission.

Hmm. thats an option as well.. but if you look at:

http://www.ietf.org/internet-drafts/draft-ietf-ipfc-fcoverip-02.txt

it specifically states that you DO NOT want retransmission...

In either case a SCTP endpoint that supported u-sctp and sctp would
be able to have retransmission turned on OR turned off, on any particular
stream... and it would be able to have one association doing both at
the same time :-> 

You basically can have it any way you want it :-)

R

> 
> -Wayland

-- 
Randall R. Stewart
randall@stewart.chicago.il.us or rrs@cisco.com
815-342-5222

From:	Wayland Jeong [wayland@troikanetworks.com]
Sent:	Thursday, August 24, 2000 10:37 PM
To:	'Marjorie Krueger'; Randall R. Stewart
Cc:	Wayland Jeong; ipfc@standards.gadzoox.com
Subject:	RE: draft-otis-fc-sctp-ip-00.txt

> It seems like people are missing the intended application of FCoverIP -
namely, a
> switching device, NOT an endpoint.  Switching devices are not aware of the
> protocol running over FC and must treat every FC packet similarly.  SCSI
is only
> one application that runs over FC.  An FCoverIP device can't selectively
decide to
> retransmit a packet without implementing some pretty complicated policies,
and
> looking into the headers of each packet, which would degrade performance
so much
> that it's questionable whether this would be a feasable application at
all.  This
> has been discussed somewhat on the IPS reflector.
> 
Layer 4 through layer 7 LAN switches exist today (Foundry, Alteon, Extreme,
etc.) that
inspect deep into each packet and make wire-speed decisions about how to
service the
data. There are routers and firewalls which terminate TCP and thus run
entire stacks in
the box. I'm not saying that it is trivial to design a port that can keep-up
with the wire and
run a session protocol in hardware, but it is not out of the question. If it
was, iSCSI would
be doomed as a high-performance gigabit solution (and that is an entirely
different argument).

I guess, what I'm trying to say is that someone should clearly state the
requirements for
FCoverIP. If the requirements are that it simply run on a private tuned
network which can
virtually guarantee a loss-less medium, then I would argue that we don't
need congestion
control and hence a retry mechanism. I would also argue that FCoverIP is a
niche, proprietary
solution which probably does not belong in IETF since it is not intended for
the internet. 

If the requirement is that FCoverIP must co-exist with other LAN/WAN
traffic, then I would
argue that at a minimum we need a congestion management scheme since the
networks
depend on it. If we have a need for congestion management that implies that
things are
getting dropped (unless QoS ensures that does not happen.). And if things
are getting
dropped, then retries at the SCSI I/O level could severely limit
performance. 

That was the convoluted reason for my comments about needing to have a retry
mechanism.
I believe that you need a reliable medium for transporting any SCSI-based
protocol.

> Marjorie Krueger
> Vixel Corporation
>

-Wayland

From:	Randall R. Stewart [randall@stewart.chicago.il.us]
Sent:	Friday, August 25, 2000 4:45 AM
To:	Wayland Jeong
Cc:	'Marjorie Krueger'; ipfc@standards.gadzoox.com
Subject:	Re: draft-otis-fc-sctp-ip-00.txt

Wayland:

My sole reason for pointing out the unreliable/non-retransmit
option of SCTP is because of a working group document.
In the document it is very very specific and states that
the WG does not like TCP because of the retransmit storms
that can occur if TCP and the FC layer both retransmit.

I simply offered a solution, and since SCTP could offer
both non-retransmit and retransmit it takes this argument
off the table...

Other comments in-line below:


Wayland Jeong wrote:
> 
> > It seems like people are missing the intended application of FCoverIP -
namely, a
> > switching device, NOT an endpoint.  Switching devices are not aware of
the
> > protocol running over FC and must treat every FC packet similarly.  SCSI
is only
> > one application that runs over FC.  An FCoverIP device can't selectively
decide to
> > retransmit a packet without implementing some pretty complicated
policies, and
> > looking into the headers of each packet, which would degrade performance
so much
> > that it's questionable whether this would be a feasable application at
all.  This
> > has been discussed somewhat on the IPS reflector.
> >
> Layer 4 through layer 7 LAN switches exist today (Foundry, Alteon,
Extreme, etc.) that
> inspect deep into each packet and make wire-speed decisions about how to
service the
> data. There are routers and firewalls which terminate TCP and thus run
entire stacks in
> the box. I'm not saying that it is trivial to design a port that can
keep-up with the wire and
> run a session protocol in hardware, but it is not out of the question. If
it was, iSCSI would
> be doomed as a high-performance gigabit solution (and that is an entirely
> different argument).
> 
> I guess, what I'm trying to say is that someone should clearly state the
requirements for
> FCoverIP. If the requirements are that it simply run on a private tuned
network which can
> virtually guarantee a loss-less medium, then I would argue that we don't
need congestion
> control and hence a retry mechanism. I would also argue that FCoverIP is a
niche, proprietary
> solution which probably does not belong in IETF since it is not intended
for the internet.
> 

If you ARE on a private net, even if Congestion Control is in place it
SHOULD never be engaged. Here a SCTP with non-retransmit will help
you. You never have the danger of a retransmit storm, and the fact
that your network IS so reliable means you never get the CC algorithms
working against you . Also if someone makes a mistake, and puts a 
private configuration across the big I-Internet it provides a
protection..


> If the requirement is that FCoverIP must co-exist with other LAN/WAN
traffic, then I would
> argue that at a minimum we need a congestion management scheme since the
networks
> depend on it. If we have a need for congestion management that implies
that things are
> getting dropped (unless QoS ensures that does not happen.). And if things
are getting
> dropped, then retries at the SCSI I/O level could severely limit
performance.

And here, no matter what the performance level, one could turn on
the reliable side transport of SCTP since it IS on a WAN and the
retransmit on the SCTP level would save you from hitting SCSI I/O
retransmits..

> 
> That was the convoluted reason for my comments about needing to have a
retry mechanism.
> I believe that you need a reliable medium for transporting any SCSI-based
protocol.
> 
> > Marjorie Krueger
> > Vixel Corporation
> >
> 
> -Wayland

R
-- 
Randall R. Stewart
randall@stewart.chicago.il.us or rrs@cisco.com
815-342-5222