RE: iSCSI:flow control, acknowledgement, and a deterministic recovery

["Douglas Otis" <dotis@sanlight.net>]

Following on,

As an overview, there is a change allowing commands trapped within the
sequencer to be rejected for replay at a new sequence.  There would need to
be a means of re-associating the PDU with new CmdSN or use the X-bit to
reissue.  The rejected sequences should become invalidated for only those
rejected due to an immediate command bypass and disallow the X-bit.  To
replay, these sequence associations would require reassignment.

The advantage would be:
  1) No need to score-board any sequencer bypass.
  2) Immediate opening of the window for urgent commands.
  3) Quick repairs for a connection problem.
  4) Immediate acknowledgement for all executed commands.
  5) No need to trap iSCSI errors within SCSI layer.

You would want to invalidate sequences only for commands rejected due to an
immediate command placement.  PDU rejected due to a digest error appears to
invalidate the sequence within the iSCSI proposal.  What is abort task
action, retire SCSI tag?  Is the target making assurances of a free tag?
Targets detecting data digest errors may abort tasks by issuing a Reject PDU
after waiting for all data if it does not support R2T. Is the only method of
control, a SCSI layer abort, and should this be an option?

Notes:
Ver 5 Page 80:

"6.2 Digest Errors

   When a target receives an iSCSI PDU with a header digest error or a
   payload digest error in an iSCSI PDU, it MUST answer with a Reject
   iSCSI PDU with a Reason-code of Header-Digest-error or Data-Digest-
   Error and discard the offending PDU.  If the error is a Data-Digest-
   Error in a Data-PDU, the target MUST either request retransmission
   with a R2T or answer with a Reject iSCSI PDU and abort the task.
   However if the error is detected while data from the initiator is
   still expected (the command PDU did not contain all the data and the
   target has not received a Data PDU with the final bit Set) the target
   MUST wait until it receives the a Data PDU with the F bit set before
   sending the Reject PDU.

   When an initiator receives an iSCSI PDU with a header digest error,
   it MUST discard it.  When an initiator receives any iSCSI PDU other
   than a data PDU, with a Data-Digest-Error, and this PDU is part of a
   task (has an Initiator Task Tag set), it MUST discard the PDU. It MAY
   restart the task (reissue the command with the same Initiator Task
   Tag and the X-bit set to 1).  If the reissued command is a SCSI
   command and it implies Read Data (Expected Data Length is not 0), the
   reissued command also includes the sequence number of the Next Data
   Packet expected by the initiator (0 if there was no data packet yet).

   When an initiator receives an iSCSI data PDU with a Data-Digest
   error, it must discard the PDU and it MUST either request the missing
   data PDUs through SACK or abort the task and terminate the command
   with an error.

6.3 Sequence Errors

   When an initiator receives an iSCSI data PDU with an out-of-order
   DataSN or a SCSI command response PDU with an EndDataSN implying
   missing data PDUs it MAY request the missing data PDUs through a data
   SACK PDU or handle this case as a connection failure.  In its turn,
   the target MUST either reject the SACK with a Reject PDU with a
   reason-code of Data-SACK-Reject or resend the data PDU.

   When an initiator receives an iSCSI status PDU with an out-of-order
   StatSN implying missing responses, it MUST either request the missing
   response PDUs through a status SACK or handle this case as a
   connection failure.  The target MUST reissue the missing responses.
   As a side effect of receiving the missing responses, the initiator
   may discover missing data PDUs. The initiator MUST NOT acknowledge
   (either explicitly through ExpStatRN or implicitly through a status
   SACK) the received responses until it has completed receiving all the
   data PDUs of a SCSI command."

Page 83:

"6.7.1.1 Recovery Within-connection

   At the initiator, the following cases lend themselves to within-
   connection recovery:

      (1)Lost iSCSI numbered Response recognized by either receiving
      it with a data digest error or receiving a Response PDU with a
      higher StatSN than expected. The initiator MAY request the
      missing responses through SACK, in which case the target MUST
      reissue them.
      (2)Requests not acknowledged for a long time. Requests are
      acknowledged explicitly through ExpCmdSN or implicitly by
      receiving data and/or status. The initiator MAY reissue non-
      acknowledged commands. The reissued, non-acknowledged commands
      MUST carry their original CmdSN and the X (retry) flag set to
      1.  Note that this is the only case in which the reissued
      command carries the same CmdSN as the "original".
      N.B. While the original connection for a command is still
      "active" (i.e., has not been logged-out or restarted), any
      command MUST be retried only on the original connection. After
      logging out the original connection, commands can be retried on
      a different connection, but must still carry the original
      CmdSN."

Doug