RE: Status summary on multiple connections

["Y P Cheng" <ycheng@advansys.com>]

> Randall and Mark:
>
> Both of you forget about the case when multiple PDUs are
> inflight, say N, N+1, and N+2, and one of them has CRC error, say
> N+1.  The receiver throws away N+1 because the bad CRC.  N+2 is
> received long before N+1 is retransmitted after a timeout by the
> sender.  Of course, a sequence number inside the PDU will ensure
> sequentialality.

OK, last time I didn't do too well with the above example.  Let me try
again.

A target device receives one TCP segment within which there are three PDUs
for commands N, N+1, and N+2.  The target ACK'ed the TCP segment and passes
it to iSCSI who parses the TCP payload.  After accepting N, the target has
to reject N+1 because of a queue-full condition.  Right at this moment, the
target finishes one previous command and now has room for one more.  Should
it accept N+2 or reject all subsequent commands until N+1 is received again?
What if N+1 never comes again?  On a network with long delay there are many
commands inflight.  My preference is letting the target accept the N+2 as
long as the application software does not mind out-of-order execution.  For
folks who design tape drives, I don't recall if we ever send multiple writes
without interleaving data. In fact, if we need write a large amount of data
to a tape drive, instead of multiple commands, we should have one command
with a very large block count.  With one command at a time, there should not
be any out-of-order execution problem.  Modern tape drives are doing "lying
writes", i.e. accepting write data immediately after the command, then
report command complete before data is written to the media.  By accepting a
write, receiving data, and report completion sequentially, there is no need
to accept more than one write command at a time.  On reading from a tape, we
could send multiple reads to keep the pipeline filled.  Then, if we drop one
read command, whoops, we are in big trouble!  Because the next read command,
N+2, -- if not rejected -- will get the data belonging to N+1.

Therefore, the only sensible solution is that the initiator should never
send more commands than the maximum queue depth allowed by the target.  This
is what we do in SCSI and 1394 adapters and should be done by a fibre
channel and iSCSI adapter.  On a network with long delay, we need a large
queue to keep enough commands inflight.

Am I OK this time?

Y.P. Cheng, CTO, ConnectCom Solutions Corp.