RE: iSCSI:flow control, acknowledgement, and a deterministic recovery

[Black_David@emc.com]

Summarizing the state of this dicussion:

(1) The idea for use of regular CmdSN numbering and
an "immediate" flag for immediate commands instead of
a zero CmdSN appears to have received a positive reception,
although the details of how CmdSN is used for immediate
commands are still open (use number in sequence vs.
duplicate what would be the next number).  Since
immediate commands are currently exempt from the
requirement to deliver from iSCSI to SCSI in order,
the default way to do this would be to exempt them
from the "MUST deliver in order" text that Doug
repeatedly quotes.

(2) It looks like some words are in order to advise
implementers to watch out for TASK_SET_FULL interactions
with CmdSN-based windowing possibly including restrictions
on issuing TASK_SET_FULL when the CmdSN window is still
open.  While the TASK_SET_FULL usage I described isn't
always present or used, there are Initiators and Targets
that behave in that fashion and hence we need some words
to advise those who plan to carry that mechanism forward
in order to avoid surprises when SCSI code (or code above
it) receives a TASK_SET_FULL.

Now the open issue:

(3) My understanding of the outstanding issue is the
rejection mechanism whereby arrival of an immediate
command with a CmdSN higher than the CmdSN of the last
command delivered to SCSI would then cause rejection
(return to sender) of all undelivered commands with
lower CmdSNs.  I think Doug is asking that support for
this mechanism be available but not be required.

The overall goal for this is apparently:

> My concern is regarding the affect [on] existing
> interfaces where there is not this potential for
> a large queue of commands that can not be purged
> except through action at the target at a much
> later point in time.

> I am asking for predictable behavior in the event
> of the sequencer being bypassed.

First of all, we need realistic motivating scenarios for
this.  The Rewind scenario isn't realistic, as Doug has
agreed that it's an application bug.  Beyond this, the
statement that existing interfaces cannot build up a large
queue of unpurgeable commands is false - if the application
issues a bunch of writes just after the Rewind, the same
situation results, and again the application is buggy.

The original motivating scenarios involved task management
commands, but I thought the following comment from Bob
Snively closed out those scenarios:

  At present, SCSI applications do not have a clear
  guarantee of the order between task management functions
  and the processing or delivery of any particular task.
  In fact, the concept of an ORDERED attribute applied
  to a task management function is unknown.  As a result,
  SCSI drivers have to be aware of the implications.

I'm assuming that Bob's comment applies to tape as well
as disk (tape experts, please correct this if it's wrong).

Absent a plausible description of what's broken, it's
hard to justify designing a fix.  There are already
situations in which SCSI behavior is unpredictable,
so a general appeal to the goodness of predictability
does not suffice - it needs to be accompanied by one
or more realistic scenarios in which such a fix would
make a visible difference.

Thanks,
--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
black_david@emc.com       Mobile: +1 (978) 394-7754
---------------------------------------------------