RE: A Transport Protocol Without ACK

["Y P Cheng" <ycheng@advansys.com>]

> From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
> julian_satran@il.ibm.com
> Sent: Thursday, September 28, 2000 2:53 AM
>
> Dear Mr. Cheng,
> I am baffled by your taxonomy. VI and I2O are basically interface (API)
> definitions and each of them has a different purpose. VI
> was ment to be a User-Space-to-User-Space communication interface and
> I2O well...a panacea for drivers.
> We are trying to build a protocol (to which an API is assumed but not
> mandated).
> Could you be a bit more specific in what you criticize or propose?

I misused the term "transport protocol" for VI and I2O.  I meant that VI and
I2O could be the API to pass an iSCSI request to an iSCSI service provider,
i.e. either an iSCSI driver above the TCP/IP layer or an adapter that
performs proper TCP/IP transport.  VI further defines connection-oriented or
connectionless for delivery of different QoS.  I2O has nothing to do with
transport.  My apology.

What I tried to point out was that this iSCSI working group is trying to
solve problems of using TCP/IP reads and writes to send PDUs that encounters
deadlock and head-of-queue problems.  If the WG can assume the use of VI-
and I2O-like API to send an iSCSI request and allow an iSCSI service
provider taking responsibility of delivery of PDUs in proper order, then,
the deadlock and head-of-queue problems are non-issues.  An iSCSI service
provider WILL send proper TCP/IP packets as well as follow the RFC2581 for
congestion avoidance.  For people who choose to implement an iSCSI provider
using TCP reads and writes, they do have an implementation problem.  But,
should the WG address the problems of every possible implementation?  We
certainly could suggest a method of using TCP/IP stack.  But, the asymmetric
model with multiple connections is not my top choice, because synchronizing
among connections is non-trivial.  I will specify an exchange table like
that in a fibre channel adapter.  We could use the initiator and target task
tags to index into the exchange table.

I am changing a fibre channel adapter to an iSCSI service provider that can
handle thousands of iSCSI requests without any queuing or resource
allocation problem.  The adapter CAN correctly interpret the iSCSI PDUs and
follow TCP/IP delivery rules including congestion control.  It also confirms
the SAM-2.  All the discussions on asymmetric-multiple-connections and
lack-of-resource problems do not apply to the implementation of this
adapter.  (I described in another posting on how the fibre channel adapters
handle thousands of requests concurrently.)

IMHO, the important discussions for iSCSI is to have a transport mechanism
that allows the service provider to stream data to the other side of the
world 10,000 miles away with 80 msec latency and 160 msec roundtrip time.
At 1Gb speed, there are 8MB of data on route to a destination in 80 msec
before the destination even has a chance to send its very first ACK back.
The originator needs to stream 16MB of data before seeing the first ACK.  We
need a lot of buffers if we implements iSCSI in TCP stack.  What if 10 Gb
InfiniBand, Ethernet, and Fibre are introduced in the next 12 months?
Obviously we can use flow control to ensure no overrun.  But, isn't it more
important to have a mechanism to move data at 10 Gb speed on an iSCSI
connection?

The WG assumes RFC2581 will handle indefinite transfer size with indefinite
delay.  I am not sure.  On a busy network, the isochronous traffic like
voice-over-IP using UDP has no ACKs for congestion control.  Hence, the
TCP/IP implementing RFC2581 will have unfair disadvantage on the Big
I-nternet.  To ensure iSCSI being a viable proposal beyond its application
to MAN where delay is small there should be discussions on the fairness on
the Big I-nternet.  Virtual circuits with paid services could be a solution.
I don't know SCTP enough to say it is the right solution.  I do understand
that the WG's charter is not to question the TCP congestion control.
Therefore, I can not raise this issue.

We have entered an era where a fibre channel adapter is capable of 30,000
IO's today and going to 50,000 and 100,000 shortly.  I don't believe the
current discussions appreciate the nature of having only 10 usec to process
an IO request.  Is it even possible to deliver 50,000 IOs using iSCSI
without discussing of mechanism of streaming 160MB of data at 10 Gb speed to
a destination 10,000 miles away?  I do believe it is possible if we can keep
the outgoing/incoming PDUs streaming with a sensible recovery of lost
packets.  But, to do this we have to have a good mechanism handling the ACK
traffic.  Introducing ACK-0 of fibre channel to deal with dropped ACK is
critical, IMHO.  However, TCP/IP is sacred and not be questioned at this
time.

Y.P. Cheng, CTO, ConnectCom Solutions Corp.