RE: iSCSI:flow control, acknowledgement, and a deterministic recovery

["Douglas Otis" <dotis@sanlight.net>]

David,

The example that I gave perhaps is not a valid case but I wanted to
illustrate which commands become invalidated by a command that bypasses the
sequencer.  Clearly, it is not the command that does the bypassing but those
commands bypassed.  My concern is regarding the affect this has with respect
to existing interfaces where there is not this potential for a large queue
of commands that can not be purged except through action at the target at a
much later point in time.  The interface offered by iSCSI is not perhaps the
same as that seen by the iSCSI to SCSI layer.

Just as you could use a non-immediate SCSI command (why exigent is a good
name.) you will also likely wish to insert additional commands, change the
reference on those issued, and the like.  With this being a potentially long
list of commands, you will not know when this list is exhausted until you
see ExpCmdSN has passed these commands.  The period of time for this process
is potentially long and will likely involve timeouts on other targets if a
tape device goes Busy.  The iSCSI sequencer could become generous as to the
concept of sequential delivery.  You imply that if staged in order
initially, then when these items are actually sent to the SCSI target layer
is not a concern.  That is not what the proposal says!  Now you have a three
minute non-immediate Rewind to wait for!

Ver 5 Pg 11.
   "The iSCSI target layer MUST deliver the commands to the SCSI target
   layer in the order specified by CmdSN."

If these commands are delivered to the target, then SCSI has the means to
handle these commands.  If stuck in the sequencer, there is NO means to
handle these commands until they poke their way through the iSCSI sequencer.
Expect this to impact the nature of SCSI handling as you suggested.

Rejecting commands within the sequencer should have zero affect on any SCSI
target.  None of these commands have ever been issued to the SCSI layer.  It
is simply a means to return these commands to the initiator to allow the
normal process of flushing.  Should there be commands to reissue, these
commands are reissued.  There is no need to reset anything.  Not requiring
commands bypassed to be rejected will cause someone to suggest a need for a
sequencer reset.  I think that an automatic rejection of bypassed commands
is the safest means to handle this situation.  Otherwise, you are going to
need to write a relaxed set of rules where sequencing happens at a point
that is never apparent or acknowledged.  The state of the system becomes
abstract.  At least with rejection, ExpCmdSN makes it extremely clear what
has been delivered to the SCSI layer.  With your scheme, you never know.

I am asking for predictable behavior in the event of the sequencer being
bypassed.  This allows for the management commands to be issued without risk
of being prevented by the command window while not being able to predict how
many commands it will take to handle these events.  Rejecting pending
commands within the sequencer at the event of a bypass should be harmless
and painless.

Doug


> > > Immediate commands are "out of sequence" by definition - immediate
> > > means deliver immediately without regard to CmdSN order.  Only the
> > > target can do this, so I don't see any problem with relying on the
> Target
> > > to do so.
> >
> > The commands that are NOW out of sequence are those commands bypassed by
> the
> > immediate command.  If this immediate command was Rewind, then
> a long list
> > of write commands become invalid as a new tape is now needed for those
> > operations.  You are expecting that the target to handle a long list of
> > invalid commands made possible by the iSCSI sequencer.  Commands
> invalidated
> > by immediate commands become out of sequence commands.
>
> And I still don't see a problem here.  Tapes have to have logic
> to deal with a write issued when no tape is loaded - this situation
> is yet another way to invoke that logic.  The target was asked
> to execute the Rewind immediately, did so, and now processes the
> writes in its new state of not having a tape loaded (presumably,
> they all result in check conditions).  Even if we do something
> clever in the iSCSI "sequencer", the same behavior can result if
> the write commands have been delivered to SCSI and queued in SCSI.
> I don't see the point of adding a new way (iSCSI rejects vs. SCSI
> check conditions) for what's essentially the same set of errors
> (tape was rewound, therefore can't be written) to manifest themselves.
>
> If the Initiator didn't want this
> behavior, it should have done something else, such as not
> using immediate delivery for the Rewind, or sending
> some suitable Task or Task Set Aborts.
>
> > I'll reserve further comment until Julian has made some progress in
> > addressing these concerns.  I also suspect it is a mistake to allow
> commands
> > to remain trapped within the sequencer during emergency or
> abnormal events
> > signified by the use of these immediate commands.
>
> Resetting the sequencer probably involves a Lun or Target
> Reset, which may be necessary in any case to clean up
> from these sort of emergency or abnormal events.  Keep
> in mind that SCSI Task Management is inherently
> non-deterministic, and so asking for completely predictable
> behavior in these sort of circumstances is just plain
> unreasonable.
>
> --David
>
> ---------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 42 South St., Hopkinton, MA  01748
> +1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
> black_david@emc.com       Mobile: +1 (978) 394-7754
> ---------------------------------------------------
>
>