iSCSI: CmdSN and Retry

["Y P Cheng" <ycheng@advansys.com>]

A few points herein would help reduce the confusion of ordered delivery
using CmdSN and error retry.

If the initiator sends both commands A and B to a target and requires B to
follow A.  After completing A with an error condition, nothing prevents the
target to start B immediately.  Therefore, what do we accomplishing by
retrying A?  In every SCSI implementation I know of, initiator is always
ultimately responsible of understanding the ordered execution as well as
retry.  Therefore, the retry corner cases pointed by Santosh simply will not
exist with properly behaved initiator, unless an iSCSI initiator will behave
differently.  Since TCP ensures ordered delivery, for an iSCSI session with
multiple TCP connections, all we need to do is to ensure the sequentiality
of CmdSN from multiple connections.  The real question is what is the
timeout value of TCP before starting retransmit?  Even if we yank a TCP
connection, it does not change the requirement of enforcing sequential
CmdSN.

I am not aware of any SCSI target allocates resource for status phase.  Once
the status is sent, all resources are released.  If the initiator times out
the status, it retries the whole command.  A target certainly can timeout
the ACK of the status before releasing all resources.  But, the target
timeout should be shorter than that of the initiator.  But, it gets really
confusing if the target TCP has a shorter timeout value than the initiator
TCP.

A header digest error is same as a missing PDU except it is detect by iSCSI,
not TCP.  Because TCP has delivered the segment, it is possible for the
receiver to quickly notify the sender to resend the erroneous header.
Again, for a target if non-zero CmdSN is enforced, it must wait for the
resend.

In conclusion, if CmdSN is enforced, a target must take the TCP transport
delivery sequentially whether there is one or more TCP connections because
the missing one could just be an ordered-queue or head-of-queue. For a
header digest error, a target can't proceed until it gets the missing header
as long as CmdSN is non-zero.  Since a target will always move to next
command as soon as it completes one with or without error, all application
software and initiators should know better not to send a SCSI request which
depends on the success of a previous request.

The exception to the above discussion is the tape I/O.  This is because the
tape devices use "lying writes".  Once write data gets into device buffer,
the device takes full responsibility of writing the data to media.  All read
data are buffered.  Another read follows an erroneous one reads the same
data.

Y.P. Cheng, CTO, ConnectCom Solutions Corp.