Re: iSCSI: Target increment CmdSN on unretryable rejects

["Julian Satran" <Julian_Satran@il.ibm.com>]

comments in text - julo

"Pittman, Joseph" <Joseph.Pittman@netapp.com> 05/01/2002 08:05 PM Please respond to "Pittman, Joseph"

To:        Julian Satran/Haifa/IBM@IBMIL         cc:        "'ips@ece.cmu.edu'" <ips@ece.cmu.edu>         Subject:        iSCSI: Target increment CmdSN on unretryable rejects

Julian,

I am working on an iSCSI target implementation, and I have a
a question about whether the target should increment CmdSN
under certain Reject conditions.

Draft 12 of the iSCSI Specification (draft-ietf-ips-iscsi-12.txt)
says the following:

  6.2 Usage of Reject PDU in Recovery

  ...
  If the task had never been active before the Reject (i.e. the
  Reject is on the command PDU), targets should not send any
  further responses since the command itself is being discarded.
  ...
  The CmdSN of the rejected command PDU MUST NOT be considered
  received by the target, (i.e. a command sequence gap must
  be assumed for the CmdSN)


This tells me that whenever I send a Reject as the only response
to a command, I should not increment CmdSN.

+++ target is not incrementing just "taking note" and sending ExpCmdSN +++

The specification also says the following:

  6.1.1 Usage of Retry

  By resending the same iSCSI command PDU ("retry")...
  an initiator attempts to "plug"... discontinuities in CmdSN
  ordering on the target end...

  When an iSCSI command is retried, the command PDU MUST carry
  the original Initiator Task Tag and the original operational
  attributes (e.g. flags, function names, LUN, CDB, etc.) as
  well as the original CmdSN.

This is the only discussion within the specification about the
steps the initiator takes to plug a command sequence gap.

So, as I understand the combined meaning of these two sections,
when an initiator receives a Reject as the only response to a
command, it must assume that the command was discarded, and it
must 'plug the CmdSN hole' by retrying the rejected request.

+++ not if it is an intermediate reject - like that of a data packet after the command got instantiated
For a new command that is true. +++

Now consider these two situations where a target may send a
Reject:

  1) Initiator issues Text command with a non-0xffffffff value
     for Target Transfer Tag (TTT).  But the target does not
     recognize or has discarded the state for the TTT value.
     So the target sends a Reject/0x9(Invalid PDU Field).

  2) Initiator issues a SNACK/Data to request retransmission
     of a DATA_IN.  But the ErrorRecoveryLevel=0 target does
     not support SNACK, so the target sends a
     Reject/0x3(Data-SNACK Reject), or a Reject/0x5(Cmd not supported)
     or a Reject/0x9(Invalid PDU Field).


In either case, the target is rejecting the command itself, so
according to section 6.1.2, the target should not increment CmdSN.
Instead, it should delay submission of subsequently received
commands with larger within-window CmdSN values, until the
initiator 'plugs the hole'.

But according to my understanding, the initiator can only plug the
hole by retransmitting the same command.  The target will only
reject this command again, so the hole won't really be 'plugged'
after all.


What am I missing here?

A) Is there a class of 'command rejects' for which CmdSN should
   be incremented?  Namely, those commands for which retrying the
   same command doesn't make sense?

B) Or perhaps the specification must allow an initiator to plug
   a CmdSN hole by sending a NOP.

+++ the interdiction on pluging is to insure that the initiator knows what he does and has enough tools to keep the intended order. For the case you describe initiator may abort the "hole" (that is allowed) on the same connection as specified under task management. Enabling NOPs is not needed+++


I hope the answer is something along the lines of choice A.
Because, I would like to avoid, for now, the whole issue of
hole-plugging and out-of-order processing of incoming commands,
by taking advantage of

  - MaxConnections=1
  - guaranteed in-order per-connection delivery
  - escalating digest errors to session recovery

Any guidance would be greatly appreciated.
--
Joe Pittman
jpittman@netapp.com