RE: iSCSI: Markers

["Douglas Otis" <dotis@sanlight.net>]

Somesh,

One view of evolution regarding TCP would be that it has already evolved
into SCTP.  There are many problems beyond framing being solved as a result.

Doug

> > In message
> > <NMEALCLOIBCHBDHLCMIJAEBICKAA.somesh_gupta@silverbacksystems.com>,
> > "Somesh Gupta" writes:
> >
> > >Jim,
> > >
> > >What are the issues with "one PDU per TCP segment"?
> >
> > Mostly that what you're asking for is no longer TCP.
> > TCP does not preserve record, or packet, or push boundaries.
>
>   Let us not be so religious about it. Everything must evolve.
>   TCP has always evolved. To create cost-effective multi-gigabit
>   adapters, I do think changes are needed to "packetize" TCP.
>   Others are proposing changes to initial window size, ECN
>   etc etc.
>
> >
> > TCP retransmissions can, and will, resend a maximal-MTU worth of
> > data from the point where a DUPACK triggers fast retramsit.
> >
> > That maximal segment contains the tail (perhaps all) of one iSCSI PDU,
> > plus as much as of the following PDU as fits inside the TCP segment.
>
>   Ok. We do require change to the TCP implementation. I already gave
>   you that.
>
> > Heck, if iSCSI can enqueue another PDU before the first one has gone
> > out the wire, then standard TCP semantics is to compact the second PDU
> > into the first one.  In the reference BSD implemetnation, see
> > sbappend().
>
>   OK. Changes are needed to the implementation. Some OSes already avoid
>   willy-nilly segmentation.
>
> >
> > SACK makes the picture more complicated. Retransmissions in the face
> > of PMTU and route/MTU changes make it even worse. But you get the idea.
>
>   This is where the determinism of COWS addresses lets you handle
>   such issues on the slow path.
> >
> > If iSCSI PDUs are bigger than TCP segments, then dropping segment
> > with a PDU boundary leaves the receiver unable to synchronize until it
> > receives the TCP retransission. That takes at least an RTT,
> > which takes us back to  BW*RTT worth of buffering.
>
>   There are many alternatives to avoid this one.
> >
> > Changing TCP, giving TCP senders the option of asking their sending
> > TCP to not coalesce segments across sender-marked boundaries, might
> > well be generally useful (c.f. the history behind PSH); but it's a
> > change to TCP. That's outside the IPS charter.  It probably requires
> > an extension to the API to TCP as well, to mark the boundaries.
>
>   Again, no protocol changes and interoperability with existing
>   implementations is the goal. Even with markers, the receive
>   TCP has to change significantly.