iSCSI Reqts: In-Order Delivery

[Black_David@emc.com]

About a week ago, Santosh Rao wrote:

> I object to the following requirement :
> 
> " MUST support ordered delivery of SCSI commands from the initiator to
> the  target, to support SCSI Task Queuing. "

In the hopes of driving this discussion to closure,
let me summarize my understanding of this situation:

- SAM2 expects ordered delivery, but does not mandate
	it.  Section 4.6.2 of SAM2r16 says:

  For convenience, the SCSI architecture model assumes in-order delivery
  to be a property of the service delivery subsystem. This assumption is
  made to simplify the description of behavior and does not constitute a
  requirement.

	I will observe that there is well-known code for disk
	initiators (e.g., staircase/elevator/etc. command queue
	reordering for seek optimization) that relies on this
	expectation for performance (not correctness).  IMHO,
	making in-order delivery purely OPTIONAL would not be
	consistent with the spirit/intent of the above SAM2 text.

- In existing SCSI transports, transport errors (e.g., CRC failure)
	can cause ordering issues at higher levels.  Both Parallel SCSI
	and Fibre Channel initially drop anything that has a transport
	error.  IF the higher level code retries AND other commands were
	executed before the error was noticed by the retry logic, THEN
	the retry occurs in a different place in the command sequence
	from the original command.  I believe that existing tape
	implementations usually either don't retry, or spoon-feed the
	tape one command at the time to make sure no other command can
	get in the way.  FCP-2 has the ability to prevent other commands
	from being executed ...

- iSCSI has to face this issue because we're incorporating a retry
	mechanism into the base protocol.  Parallel SCSI and FCP-1 could
	ignore this because they didn't do retries.  FCP-2 has a retry
	mechanism, but makes order preservation of retries negotiable -
	order of command delivery in the face of retries is only assured
	if CRN is in use, and that is controlled by a settable bit in a
	mode page (FCP2r7, 4.3).  The Initiator can choose not to set
	this bit, and the Target can refuse to allow the Initiator to set
	it.

- CRN is a relatively new mechanism, and both it and the associated
	FCP-2 retry mechanism are likely to be implemented for/used by
	tapes.  Disks will still have an expectation of in-order delivery
	in some cases, even though neither of these mechanisms are in use.
	IMHO, Santosh's proposal to use CRN for all
expectations/requirements
	of ordering doesn't seem like a good idea for this reason.

So, let me try the following proposal to resolve this issue/objection:

(1) MUST provide ordered delivery of SCSI commands from
	the initiator to the target in the absence of transport
	errors visible to iSCSI (e.g., iSCSI CRC failure,
	unexpected TCP connection closure).
(2) MUST specify the ability to preserve ordered delivery
	of SCSI commands even in the presence of transport
	errors.  A mechanism MUST be provided to allow
	Initiators and Targets to negotiate this preservation
	on a per-session or finer granularity basis, 

The Rationale for (1) is the combination of the SAM2 expectation
plus the fact that there are situations in which disks expect
this ordering in the absence of mechanisms like CRN that can
enforce it.  The Rationale for (2) is to provide support
analogous to FCP-2 - this should be sufficient for tapes to
obtain the behavior they require.

Comments?

Thanks,
--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
black_david@emc.com       Mobile: +1 (978) 394-7754
---------------------------------------------------