RE: iSCSI: "conservative reuse" requirement

[Black_David@emc.com]

> Thanks for the message.  Let me comment on what I agree with first.
>
> >if "conservative reuse" does not
> > happen (breaking shared persistent reservations), the reason that
> > it does not happen MUST be externally configurable as opposed to
> > being an unchangeable internal characteristic of the implementation.
> 
> Agreed.  If I understand you correctly, you're concerned that NICs
> that are hard-coded not to allow single initiator port abstraction (i.e.
that
> don't do conservative reuse) will break the new shared persistent
reservations.
> I agree that it currently is an issue, I propose that we have a NIC
requirement
> in the implementation notes (like the node name configurability
requirement)
> that says something like - Every iSCSI NIC MUST provide a means of
> enabling a strict "conservative reuse" of ISIDs across different target
portal
> groups.

Good.  Once that's done, what's the value in permitting operating modes
other than "conservative reuse"?  The risk of multiple operating modes is
that there'll be:
- The mode that works, and
- The mode that complies with the standards.
Having only one mode avoids this problem.  Don't laugh too loudly -
versions of this situation are occurring in Fibre Channel switches
today.

> With that said, here's the idea (repeated in several places) 
> that I have a problem with -
> > every possible connection is
> > attempted, and the reasons that not every connection is established
> > are external (and configurable, modulo physical constraints).
> 
> First off, I am not sure what you meant by "connection" here, and
"presented"
> in your previous message.  I took it that you meant "establishing a SCSI
nexus".
> If that's indeed so, the assertion that it's only the external factors
that decide
> nexii is not always true.  Consider the case of an FC initiator 
> port discovering target ports using an FC Name Server, and an iSCSI
initiator
> port discovering target ports using a "SendTargets" command.  In either
case,
> the initiator SCSI port may not really establish a nexus with a discovered
> target SCSI port, unless the application (or
> even the SCSI class driver) really wants to do an I/O (starting with an
open()
> system call in Unix systems).

I strongly disagree with that.  Virtually all commercial OS implementations
of SCSI do aggressive establishments of the nexii as part of the boot
sequence,
as SCSI code tends to expect a "bus walker" to find devices in the boot
sequence, and make sure they work.  Put a trace analyzer to work on a boot
and watch all the TEST UNIT READY and similar commands flow by.  Is anyone
making significant changes to this "bus walker" boot paradigm out there?

There's also a subtle issue to watch out for here in that this delays
error discovery - if the shared persistent reservations are being
used by cluster software, discovering that the alternate path nexus can't
be established when trying to fail over to it is way too late.  For things
like quorum volumes, I'm fairly sure that cluster software checks all the
access paths at initialization time, but can someone familiar with cluster
software comment on whether the alternate access paths needed for failover
of application volumes are checked in advance, vs. relying on the OS to
complain if the SCSI connection to the device didn't set up correctly?

> But in any case, it appears to be in the implementation territory.

That's something I can agree with - namely that when nexii are established
is an orthogonal issue to the session identifiers used to establish
them, although I would caution implementers that there's a lot of code
out there that operates under the "bus walker" paradigm and hence wants
to see nexii (sessions) established aggressively.  In a code stack that
doesn't operate under the "bus walker" paradigm, "conservative reuse"
is still applicable, as it need only constrain what ISIDs have to be
used, and not when they are used.

Taking a whack at requirements, I think there are at least two of them:
- MUST support "conservative reuse" without requiring any iSCSI sessions
	to be terminated in order to set up the required sessions (i.e.,
	one can take the ISID from an existing session and a target portal
	group to which that ISID is not in use and set up an iSCSI session
	with that ISID without having to terminate any existing session for
	ISID usage reasons - this is tricky to state precisely when the
	possibility of allowing sessions to be set up on the fly well after
	booting is allowed).
- SHOULD use "conservative reuse" in setting up sessions from the same
	Initiator Portal Group (set of IP addresses) to different Target
	Portal Groups (i.e., reuse an existing ISID in preference to
	creating a new one).
I presume the "mapping police" will let us know if Initiator Portal Group
is not the right term for the second requirement ;-).

Thanks,
--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 249-6449 *NEW*      FAX: +1 (508) 497-8500
black_david@emc.com         Cell: +1 (978) 394-7754
---------------------------------------------------