RE: iSCSI:flow control, acknowledgement, and a deterministic recovery

["Douglas Otis" <dotis@sanlight.net>]

Julian,

How many such "immediate" commands do you expect to support?  Would you need
to see the status returned from the first before sending the next?  By
copying the CmdSN of the previous non-immediate command, you forgo
acknowledgement.  Although this does relieve the sequencer the problem of
patching holes, it comes at the expense of no longer having any flow control
or acknowledgement.  If this interface supported just a single target, then
a single such immediate command may be appropriate.  As this interface
supports many such targets, rejection of commands pending within the
sequencer seems far more appropriate in allowing the needed latitude in
error conditions.  Your scheme seems to expect a methodical technique of
sending a command, waiting for status, sending the next etc.  If the problem
was indeed urgent, then rejecting these bypassed commands would be far more
responsive, provide acknowledgement, flow control, not depend on the target
model handing such out of sequence commands, and not requiring any spare
resources to be maintained.  To me, this subject seems to merit discussion
if only to modify the zero sequence to a copy of the previous command CmdSN
with a flag added.

Doug

> David,
>
> Resource "stress" is the same as for immediate marked with 0 - as
> immediate
> commands are not
> controlled by a window in any case.
>
> The "new scheme" might use a flag and the CmdSN field will carry the next
> CmdSN (immediate commands will not cause the counters to advance - but 0
> does not advance either).
>
> The effect you get is that the target will have a "reference point" in the
> ordered stream where it was supposed to act.  For some commands that could
> be important for other s not.
>
> Obviously the immediate commands get ordered with reference to
> non-immediate commands but not between themselves if issued without
> intervening ordered commands (a partial order).
>
> Obviously I will have to reformulate the "rules of engagement" for command
> retry but I think it worth doing.
>
> And yes - there is no free lunch - I will have to put in some work -:)
>
> Thanks for keeping the subject up long enough for me to get the feeling
> that it might be a problem (and simple solution) out there.
>
> Regards,
> Julo
>
> Black_David@emc.com on 09/04/2001 17:11:44
>
> Please respond to Black_David@emc.com
>
> To:   Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu
> cc:
> Subject:  RE: iSCSI:flow control, acknowledgement, and a
> deterministic reco
>        very
>
>
>
>
> > The main reason for selecting 0 and not a flag for immediate
> delivery was
> > to enable immediate delivery even when the command window is closed.
> >
> > However we can achieve the same effect by using an immediate flag and
> > using the current CmdSN without advancing it.  With this we get a
> > reference to where in the stream the command where supposed to
> act if the
> > stream order is important.
>
> I think this causes resource management complications on the target
> because reusing the CmdSN for an immediate command requires a new
> command buffer, unless we allow the target to arbitrarily
> reject/abort some
> command to make room, which seems wrong (the immediate command
> may not be a task management command, and even then arbitrary
> rejects/aborts may not be desirable).
>
> How is this better than requiring the Initiator to not permit the command
> window to close (i.e., the Initiator always keeps one slot in the window
> open for one immediate command, or N slots for N immediate commands,
> or no slots if it wants to live dangerously)?  I think existing (SCSI and
> FC)
> Initiators have to do this sort of resource management anyway, as I don't
> believe immediate commands are exempt from TASK_SET_FULL.
>
> There appears to be a "no free lunch" principle here in that some piece
> of the iSCSI Initiator-Target system somewhere has to be cognizant
> of the fact that resources for immediate commands are being
> reserved on the target, and Initiator control of the command window
> (coupled with a requirement that MaxCmdSN never decrease) seems
> to be the easiest way to get there.
>
> --David
> ---------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 42 South St., Hopkinton, MA  01748
> +1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
> black_david@emc.com       Mobile: +1 (978) 394-7754
> ---------------------------------------------------
>
>
>
>
>