RE: iSCSI:flow control, acknowledgement, and a deterministic recovery

["Douglas Otis" <dotis@sanlight.net>]

Santosh,

In a change recommending a flag and a valid CmdSN and not a null value, I
included a flag named "Casual" to provide this mode of operation you
describe which still allows flow control.  I would not wish to permit this
Casual mode, but as David feels such lists or threads are easy to manage,
this should not represent any greater effort for feedback.  But then again,
if you are not using flow control, you are also not using any
acknowledgement other than an eventual SCSI status.  If there is no flow
control, this removes resource control and reception acknowledgement.  If
this is not needed, then such flow control should be removed for simplicity
as there then must always be an abundance by design of adequate recourses.
For cases where command sequence is important, CmdSN together with
acknowledgement must remain or only one command at a time can be issued.  I
can envision three modes, a Sequential mode, a Casual mode, and an Exigent
mode where the command is given head of sequence status (with rejection for
feedback simplicity ;).

You seem to be placing many things on the table for reconsideration.  Flow
control, acknowledgement, sequential delivery, and perhaps even a means of
moving a command up the queue feeding SCSI targets.  One command at a time
for sequential assurance is not impossible.  Is flow control needed?  Do you
care if there is any reception acknowledgement if all feedback is in the
SCSI layer?  It would seem in this barebones case, iSCSI adds little to
ensure integrity.  The FC encapsulation places a time stamp within the PDU.
Perhaps a time stamp should be employed as an alternative to sequence
control?  At least after a timeout, you are assured a missing command is
forever gone.  Otherwise, the check was to see that CmdSN is not too small
to ensure it is not a stray.

Doug


> Doug [& All],
>
> Some comments on this thread :
>
> 1) The immediate command feature can be exploited by initiators which
> are not required to provide strict command ordering for their SCSI
> sub-system. (The majority of today's scsi stacks). IOW, all commands can
> be sent with CmdSN=0 to indicate no ordering required.
>
> Any scheme to restrict the number of "immediate commands" that can be
> sent prohibits such a feature and is un-desirable.
>
> 2) The (ExpCmdSN, MaxCmdSN) based acknowledgement and flow control is a
> freebie that targets can use to implement additional flow control.
> There's nothing in the spec that mandates that a target MUST use this
> feature to throttle its window and apply flow control.
>
> Hence, any dependence and assumptions on the existence of CmdSN based
> flow control may be incorrect, since the target may be using QUEUE FULL
> (TASK SET FULL) to apply flow control.
>
> > I also suspect it is a mistake to allow commands
> > to remain trapped within the sequencer during emergency or
> abnormal events
> > signified by the use of these immediate commands.
>
> 3) If the initiator suspects that the CmdSN queue at the target is
> stuck, it can always use a task mgmt command or even an iSCSI login with
> the X (restart) bit to perform a cleanup of stuck commands at the
> target. There's no need to build in implicit assumptions that a CmdSN
> with the immediate flag result in a clean-up of previously pending
> commands at the CmdSN queue of the target.
>
> - Santosh
>
>
> Douglas Otis wrote:
> >
> > David,
> >
> > > > To not support rejects, you are then relying on the target to
> > > > handle these now out of sequence commands.
> > >
> > > Immediate commands are "out of sequence" by definition - immediate
> > > means deliver immediately without regard to CmdSN order.  Only the
> > > target can do this, so I don't see any problem with relying
> on the Target
> > > to do so.
> >
> > The commands that are NOW out of sequence are those commands
> bypassed by the
> > immediate command.  If this immediate command was Rewind, then
> a long list
> > of write commands become invalid as a new tape is now needed for those
> > operations.  You are expecting that the target to handle a long list of
> > invalid commands made possible by the iSCSI sequencer.
> Commands invalidated
> > by immediate commands become out of sequence commands.
> >
> > I'll reserve further comment until Julian has made some progress in
>
> >
> > Forgive my advocacy, but if one does not attempt to support a
> position, then
> > the subject is not explored.
> >
> > Doug