iSCSI: I/O (command) recovery

[Matt Wakeley <matt_wakeley@agilent.com>]

There's been some confusion on how iSCSI commands that fail could be recovered
at the iSCSI layer without involving SCSI, so I thought I'd write up (my
understanding of) how this is done.

For a Target Read operation:

1- The initiator sends a read command to the target.

2- The target iSCSI sends the command to the SCSI layer.

3- The SCSI layer performs the read command and hands the read data and status
to the iSCSI layer. iSCSI then allocates resources and stores the read data
and the I/O status in the allocated resources.

4- The target iSCSI sends the read data and status to the initiator, but does
*not* deallocate the resources associated with the I/O.

5- When the initiator iSCSI successfully receives the data and status, it
acknowledges the receipt of the status back to the target iSCSI via the
ExpStatSN.

6- When the target iSCSI receives the ExpStatSN acknowledging the I/O, it then
deallocates the resources for the I/O.

If some error occurs such that the initiator iSCSI does not receive all the
data and status, then the initiator iSCSI performs whatever
link/connection/session recovery to re-establish communication with the
target.  At that point, the initiator iSCSI layer re-issues the I/O with the
retry bit set.

- At this point, the target iSCSI layer (*not* the SCSI layer) recognizes the
command is a duplicate (because the retry bit is set), and instead of sending
the command to the SCSI layer, it simply retransmits the buffered data and
status [continue at #5 above].

For a Target Write operation:

1- The initiator sends a write command to the target.

2- The target iSCSI sends the command to the SCSI layer.

3- The SCSI layer prepares for the write and asks the iSCSI for the data.
iSCSI then allocates resources for the I/O.

4- The target iSCSI sends R2T to the initiator.

5- The initiator sends the write data to the target.

6- The target iSCSI receives the data and sends it to the SCSI layer.

7- The SCSI layer performs the write and gives the status to the iSCSI layer.

8- The target iSCSI sends the status to the initiator (and saves it in the
iSCSI allocated resources for the I/O), but does *not* deallocate the
resources associated with the I/O.

9- When the initiator iSCSI successfully receives the status, it acknowledges
the receipt of the status back to the target iSCSI via the ExpStatSN.

10- When the target iSCSI receives the ExpStatSN acknowledging the I/O, it
then deallocates the resources for the I/O.

If some error occurs such that the initiator iSCSI does not receive the
status, then the initiator iSCSI performs whatever link/connection/session
recovery to re-establish communication with the target.  At that point, the
initiator iSCSI layer re-issues the I/O with the retry bit set.

- At this point, the target iSCSI layer (*not* the SCSI layer) recognizes the
command is a duplicate.

- If the target iSCSI has not received all the data for the R2T, then the
iSCSI layer re-issues the R2T [continue at #5 above].

- If the target iSCSI layer has sent the SCSI status, it simply retransmits
the status [continue at #9 above].

Now, this is just a basic overview on how it could be done.  I'm sure there
are variations on how it could be implemented in the target. Should this
example be put into the iSCSI document?

-Matt Wakeley
Agilent Technologies