RE: iscsi: rev 05 2.5.1 requires ACA support

["Charles Binford" <Charles.Binford@BlueSpruceNet.com>]

(sorry this got kind of lengthy - but I do have proposed solution to what I
believe are the problems below)

Julian,
I don't think iSCSI needs to look at the NACA bit in the CDB - I don't think
iSCSI should care about ACA at all.

In my 10+ years of working with SCSI disk products I've yet to see a host
driver that cared about ordering.  As David said below, if there is an
ordering dependency then the application waits for completion of the first
I/O before releasing the second.  One gets the best performance from a SCSI
block I/O target (from a single disk to a large RAID controller) by issuing
queued commands with simple tags and allowing the target to freely re-order
the commands internally.  When an error happens it doesn't matter what
commands are in-flight because they are all independent I/Os.  This is the
way high performance SCSI disks and disk subsystems work.  It would be a
mistake (in my opinion) for iSCSI to come in and mandate the extra
complexity of ACA to handle a corner case most of the SCSI world it is
trying to penetrate doesn't care about.

Granted, tape and other sequential devices are a different story.  However,
I participated heavily in T11 for 2 to 3 years helping develop a error
recovery scheme that would allow SCSI tape devices to work with FC.  The
bottom line, however, was that while we talked about it over and over, no FC
tape vendor had a device that supported queuing.  If you don't queue, then
nothing is in-flight when an error occurs.  (streaming is accomplished by
caching in the target).  After this drug on for a couple of years and the
complexities of queuing to sequential devices became evident, a new project
was started in T10 to develop a new tape command set that included relative
block addresses in the CDB - making the new tape model very similar to
disk - each command can be interpreted independently and precise delivery
order is not a fundamental requirement.

===================
OK, enough of my preaching.  Here is my understanding of the issues, and
what I believe the solution set should include:

Problem Statements:
--------------------------
- problem 1) When an initiator's I/Os are cleared by it own actions (e.g.
sending a task management function of abort task set) how does it know which
of issued commands were aborted and which were still in-flight and had not
yet made it to the target.

- problem 2) (closely related to 1)  How are task management functions
delivered - with the immediate delivery option (CmdSN=0) or not? (reference
1.2.2.1, top of page 12)  If immediate, then problem 1 is aggravated if
multiple connections are used.  If not immediate, then we can have a problem
with the task management function never being delivered because the I/O we
are trying to abort never made it to the target and the target's iSCSI layer
wont' pass up the task management function because the CmdSN number is
wrong.

- problem 3) When an initiators I/Os are cleared by the actions of another
initiator (e.g. the other imitator sent clear task set or lun reset) how
does it know which of its outstanding commands were cleared and which will
finish.

Proposed Solutions:
------------------------
+ solution to 1 and 2) I propose all task management functions be required
to use the  following:
  - immediate delivery option

  - utilize a new task management function sequence number field (call it
TskMgmtSN)

  - all task management functions shall be sent on all active connections
with the same value for TskMgmtSN

  - the target must receive the same task management function (identified by
TskMgmtSN) on all active connections before it acts on it.  (Use a timer
(TBD) to weed out and close down defective connections.)

  - the target must send the task management response on all active
connections, again identified by the TskMgmtSN assigned by the initiator.


+ solution to problem 3)  Two currently defined mechanism in SCSI already
take care of this.  Neither are widely used today, but they are already
defined.

  A) initiator may set NACA on all commands and the target will enter ACA
when it sends the unit attention that commands were cleared by another
initiator.  (Note, I see no need for the async event notice.)

OR

  B) enable the newly defined Task Aborted status.  This option doesn't give
any guarantees about ordering, but as I argued above the vast majority of
SCSI devices don't care.


Charles Binford
Blue Spruce Networks
office: (316) 315-0382 / cell: (316) 210-6404
e-fax: (509) 756-4425




-----Original Message-----
From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
julian_satran@il.ibm.com
Sent: Friday, March 16, 2001 9:33 AM
To: ips@ece.cmu.edu
Subject: RE: iscsi: rev 05 2.5.1 requires ACA support




David,

As SCSI can liberally set or reset the bits in CDB and we don't want iSCSI
to have to look into it (we managed to avoid this until now) I though that
it will serve us well to request ACA support so that we get it at least
from all the new drivers.

As with many of other tiny inconsistencies we are bound to encounter this
stems in part from the fact that SAM considers the specific protocol (FCP,
SBP etc.) as a blending of transport and SCSI (see the many protocol
specific parameters and behaviors) without specifically addressing the
"separability" issue that the layering purists among us demand.

Regards,
Julo


Black_David@emc.com on 16/03/2001 16:21:53

Please respond to Black_David@emc.com

To:   Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu
cc:
Subject:  RE: iscsi: rev 05 2.5.1 requires ACA support




If that's the case, then the wording that Ralph
pointed out needs to be modified to indicate that
ACA is used only when appropriate.

--David

> -----Original Message-----
> From:   julian_satran@il.ibm.com [SMTP:julian_satran@il.ibm.com]
> Sent:   Friday, March 16, 2001 2:34 AM
> To:     ips@ece.cmu.edu
> Subject:     RE: iscsi: rev 05 2.5.1 requires ACA support
>
>
>
> We are aware of the support for ACA missing from some drivers.
> The situation is even exacerbated by the fact that certain exceptions
that
> are not errors per-se
> will require ACA to be fired to accomodate for commands in flight (like
> reservations, busy, task-set-full).
>
> However actions at the initiator can be fine-tuned with the NACA bit in
> CDB
> and the ACA atrribute for the task.
>
> Julo
>
> Black_David@emc.com on 16/03/2001 03:46:29
>
> Please respond to Black_David@emc.com
>
> To:   Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu
> cc:
> Subject:  RE: iscsi: rev 05 2.5.1 requires ACA support
>
>
>
>
> > After an error if you have commands in flight you want them all dropped
> > until you specifically reset the ACA and restart the queue (prevent
> things
> > to be executed out of order).
>
> The T10 folks will have to go after this one in more detail, but Julian's
> above statement is correct *only* if the commands in flight depend on
> the one that caused the error (i.e. executing them out of order will
cause
> problems).  This is generally not the case for disks where the usual
> practice is to enforce command execution order dependencies
> (e.g., database write ordering) in the operating system and applications
> by waiting for responses (yes it's possible to do better, but lots of
> existing software doesn't).  The result is that commands in
> flight can be executed in arbitrary order with arbitrary ones of them
> failing without causing further difficulties.  As Ralph has pointed out,
> most hosts do not use ACA for disk-based storage, and if iSCSI
> always does ACA, this will cause nasty integration issues.
>
> Just to stir the pot further, I believe Fibre Channel provides a negative
> example, because if FC drops a frame (which is not a good idea,
> but can still happen), the FC component that dropped the frame has no
> clue about what ACA is, or how to get the target (which is not the
> same piece of hardware) to enter ACA state.  Both Class 2 and Class 3
> are vulnerable to this.
>
> Tapes are another matter - do we still have a tape expert on the list?
>
> I thought where we were headed on ACA was something along the
> lines of:
> - Targets MUST implement.
> - Initiators MAY use.
> - Initiator support for ACA is NOT REQUIRED.
> which would imply a text key for Initiators to tell Targets
> whether ACA behavior is expected.  Did I miss something?
>
> --David
> ---------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 42 South St., Hopkinton, MA  01748
> +1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
> black_david@emc.com       Mobile: +1 (978) 394-7754
> ---------------------------------------------------
>
>
>