Re: iSCSI: proposal to solve the ISID issues

["Mallikarjun C." <cbm@rose.hp.com>]

Jim,

Thanks for the quick response.

As much as I would like to avoid consuming more bandwidth
on this topic, let me make one last strawman for wording below.

> What words do you suggest?
...
> And a lot depends on how
> functional the NICs are: (a) just network nics, (b) protocol nics, but not
> session coordinating nics (c) full session coordination nics (with driver
> assist) (d) fully autonomous session nics.   How do we make a requirement
> that fits all the cases correctly?

I have been implying iSCSI NICs in all my postings so far -
sorry, may be that wasn't very clear.  "iSCSI Node name" comment 
obviously wasn't targeted at simple network NICs.  So clearly (a) 
is out of discussion.  But you are right - I wasn't very precise. 
Here's a strawman -

   All iSCSI hardware implementations (as in NICs) MUST allow 
   the iSCSI Node Name to be externally configurable.  All iSCSI 
   initiator hardware implementations MUST in addition also 
   support the the operational ISID values to be (either statically 
   or dynamically) externally configurable. 

> Making them configurable (at
> initialization time, e.g., as a range of values) or making them dynamic
> (the driver generates them on demand) both fit the mold.  So, I don't think
> this one is a hard requirement.

I agree that my earlier "range" wording was too constraining.
My point though is that by banning a duplicate nexus (that an
inadvertently re-used ISID allows), we are  making the ISID
(dynamic/static) distribution an imminent hard requirement
for all NICs in a given iSCSI initiator Node - except it is 
not sufficiently explicit.  Please consider the above proposed 
wording.

> Well, I don't think so.  If each vendor implements "conservative reuse"
> within their own ISID namespace on their own NICs, then you get this
> property system wide.

You are right, I guess I briefly overlooked the fact that ISID 
includes the vendor identifier within, per the new proposal.

I am personally okay with making "conservative reuse" a 
SHOULD.

Regards.
-- 
Mallikarjun 


Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions Organization
MS 5668	Hewlett-Packard, Roseville.
cbm@rose.hp.com


Jim Hafner wrote:
> 
> Mallikarjun,
> 
> Comments in line below
> 
> Jim Hafner
> 
> "Mallikarjun C." <cbm@rose.hp.com>@core.rose.hp.com on 10/29/2001 12:17:49
> pm
> 
> Please respond to cbm@rose.hp.com
> 
> Sent by:  cbm@core.rose.hp.com
> 
> To:   Jim Hafner/Almaden/IBM@IBMUS
> cc:   ips@ece.cmu.edu
> Subject:  Re: iSCSI: proposal to solve the ISID issues
> 
> Jim,
> 
> Thanks for the excellent summary.  Some questions/comments -
> 
> - I assume that the NICs must enable the configuration of
>   iSCSI Node name even in this proposal.  If it is so, please
>   call this out as a hard requirement in the main modelling
>   discussion.
> 
> <JLH>
> This requirement doesn't change from before (but how it gets written into
> the spec may differ -- and we've had this discussion before).  If a vendor
> doesn't allow configuration of his NIC's iSCSI Node name, then his NICs
> will be acting as their own iSCSI Node (that is, not representing the
> system they live in).  One can argue that this is a legitimate
> implementation (just as writing user-level code that uses its own iSCSI
> Node name).  So a "hard requirement" may be a bit too strong.  However, I
> will go with the will of the group on this point.
> 
> What words do you suggest?
> </JLH>
> - In the case of multiple NICs from the same vendor operating
>   in an end node, it appears to me that the proposal implicitly
>   assumes an ISID-range to be configurable among those NICs
>   (perhaps in vendor-unique ways, which is always what I expected).
>   If this is a correct inference, there is again a hard
>   requirement on the NICs to make the ISID range configurable.
>   Please comment on any subtle qualitative change from the
>   earlier situation that I may be missing.
> 
> <JLH>
> Well, the point is that the vendor can manage his own namespace for his
> NICs anyway he/she/it wants to.  Making them configurable (at
> initialization time, e.g., as a range of values) or making them dynamic
> (the driver generates them on demand) both fit the mold.  So, I don't think
> this one is a hard requirement (though that is probably how most vendors
> will implement their NICs).  In effect, the proposal gives vendors more
> flexibility in their own space, without causing heterogenous
> interoperability problems within the host.  And a lot depends on how
> functional the NICs are: (a) just network nics, (b) protocol nics, but not
> session coordinating nics (c) full session coordination nics (with driver
> assist) (d) fully autonomous session nics.   How do we make a requirement
> that fits all the cases correctly?  You clearly have in mind a specific
> level of implementation within the NICs, but that may not be everybody's
> model.
> </JLH>
> 
> > (2-e) Its debatable whether "conservative reuse" is a MUST or a
> > SHOULD.  My personal opinion is "SHOULD", because many systems,
> > particularly low-end that don't use reservations, can function more or
> > less OK without it.
> 
> It seems we're attempting to set ourselves up for future in
> discussing the above requirement.  Some questions -
>         - It appears to me that the "conservative reuse" can not
>           be enforced for initiators hosting NICs from different
>           vendors (since the proposal allows ISID namespaces to
>           be totally non-overlapping between non-homogenous NICs).
>           Is this a fair assessment?
> <JLH>
> Well, I don't think so.  If each vendor implements "conservative reuse"
> within their own ISID namespace on their own NICs, then you get this
> property system wide.  As before, by owning their own piece of the ISID
> namespace, they can implement what they want.  So, you may have a situation
> where some of your installed nics have this property and some don't.
> You'll find out (if your environment really needs conservative reuse) that
> you haven't got it, and it becomes a marketing/purchase spec requirement.
> </JLH>
>      - Do you see a particular disadvantage for low-end systems
>           if it's mandated (aside from the fact that they may be able
>           to live without it)?
> <JLH>
> No, but I don't see any way to really enforce it (or even really test it).
> It's not a requirement of the protocol, per se.
> </JLH>
>      - Do you see any corner cases not being met (for future)
>           if we don't make it a MUST (since you said "more or less OK")?
> <JLH>
> No, I can't see that far into the future! :-{).  One reason I'm being cagey
> here is that I'm finding it difficult to find the right words to specify
> this as a hard requirement (but I'm no technical writer either!).
> </JLH>
> 
> Regards.
> --
> Mallikarjun
> 
> Mallikarjun Chadalapaka
> Networked Storage Architecture
> Network Storage Solutions Organization
> MS 5668   Hewlett-Packard, Roseville.
> cbm@rose.hp.com
> 
> Jim Hafner wrote:
> >
> > Folks, (David Black in particular).
> >
> > Apologies for the long note, but the issue is complicated.
> >
> > The NDTeam have a proposal to resolve the concerns surrounding use of
> > ISIDs as essential components (along with iSCSI Initiator Name) of
> > SCSI Initiator Portname.  (This was rooted in private discussions
> > between John Hufferd, Julian and myself -- and came about as a result
> > of the lengthy (and boring) discussions mostly myself and David
> > Black.)
> >
> > There are two somewhat orthogonal issues involved in this discussion:
> >
> > 1) How to coordinate ISIDs in the initiator to minimize the risk of
> > accidentally breaking the ISID RULE (no parallel nexus) when
> > independent vendors co-exist in the same OS.
> >
> > 2) What ISID "reuse" rules should be specified to facilitate current
> > and future (soon?) SCSI reservation semantics (and also internal OS
> > views of SCSI Initiator Ports).
> >
> > To address these two issues, we (NDT) propose the following:
> >
> > 1) Enlarge the ISID field in the Login Request and Response to 6bytes
> > and structure it with a component that corresponds to a "naming
> > authority" (essentially the vendor generating the ISID).  So vendors
> > each have a piece of the ISID namespace to work with their own
> > components (HBAs, SW, etc). More details below.
> >
> > 2) ISIDs (within the namespace of a given iSCSIName) SHOULD be used as
> > conservatively as possible ("conservative reuse").  What this means is
> > that a given ISID should be reused for as many sessions (across
> > multiple target portal groups) as possible (but never to the same
> > target portal group twice -- that's the ISID RULE).
> >
> > NOTES and ADVANTAGES:
> >
> > (1-a) Each vendor owns his own piece of the ISID namespace, so in
> > effect, at the vendor-level, this provides a "partitioning" of that
> > namespace.
> >
> > (1-b) We don't need to specify an OS infrastructure requirement for
> > configuration of ISIDs -- each vendor can do it any way it chooses
> > (within its naming authority).
> >
> > (1-c) Breaking of the ISID RULE will only occur if a vendor messes up
> > its own implementation.
> >
> > (1-d) This is not fundamentally different from David Black's
> > suggestion for a "key"; it just (a) defines it precisely and (b)
> > embeds it in an already defined (but enlarged) field.
> >
> > (1-e) Looking towards the future, as common ISID coordination
> > mechanisms are implemented between vendors (say, SNIA banding together
> > to define a precise API), this "naming authority" can be elevated to
> > the coordinating entity or even the OS vendor.
> >
> > (1-f) ISIDs have no global uniqueness property.  They need only be
> > unique between a named iSCSI initiator and iSCSI target portal group.
> > That means that a vendor implementation can use the same algorithm to
> > generate ISIDs (under its naming authority) in every machine (OS).
> >
> > (2-a) If a SCSI target (or logical unit) is holding state (say
> > persistent reservation) for a SCSI Initiator Port (named by iSCSI Name
> > and ISID), and the associated nexus/session is dropped for some
> > reason, "reuse" by that initiator of that ISID will restore to the
> > resulting new session that state (with no other action on the part of
> > the upper SCSI layers).
> >
> > (2-b) "Reuse" of an ISID on different sessions to (necessarily)
> > different SCSI Target Ports (iSCSI Target portal groups), will enable
> > the SCSI target/logical unit to recognize a common SCSI Initiator Port
> > for those two paths.  This facilitates future changes to SCSI
> > reservations (at least).
> >
> > (2-c) An initiator driver implementated with "conservative reuse" can
> > present to the OS a stable and fairly static view of the SCSI
> > Initiator Ports (one for each ISID).  Current OS driver stacks
> > (including current wedge drivers) that are built on the presumption of
> > such a stable view, will not need modification to handle the more
> > dynamic nature of iSCSI's SCSI Initiator Port. [Note: the existence of
> > a SCSI Initiator Port presented to the OS does not *require* that this
> > port discover all possible targets; what the initiator builds sessions
> > to using a specific ISID will determine what is presented to the OS as
> > "devices" and LUs visible to that SCSI Initiator Port (could be none
> > if that ISID is never actually used!).]
> >
> > (2-d) (Related to (2-c)): Adding a new (pseudo-static) SCSI Initiator
> > Port is as simple as configuring another ISID!
> >
> > (2-e) Its debatable whether "conservative reuse" is a MUST or a
> > SHOULD.  My personal opinion is "SHOULD", because many systems,
> > particularly low-end that don't use reservations, can function more or
> > less OK without it.  This is an open question.
> >
> > DETAILS:
> >
> > 1) ISID Structure:
> > The 6byte ISID structure would be split into three parts:
> >    "type" field (1byte) -- identifies the format of the naming
> >                            authority field
> >    "naming authority" field (3bytes) -- identifies the vendor (or
> >                            group of vendors) generating this ISID
> >    "qualifier" (2bytes) -- the free-space for ISID uniqueness
> >
> > The type field takes on three defined values with all other values
> > reserved:
> >          Type    naming authority format
> >          00h     IEEE OUI
> >          01h     IANA Enterprise Number (EN)
> >          02h     "Random"
> >
> > The first types two provide a mechanism to uniquely (world wide)
> > identify the naming authority.  A vendor with one or more OUIs and
> > possibly also Enterprise number MUST use at least one of these numbers
> > when it generates ISIDs.
> >
> > The "Random" type is for the case where the ISID generator (SW or HW)
> > is provided by an entity that has no OUI or EN.  This includes, for
> > example,
> > -- a user-written program that builds sessions (and has access to the
> > system level iSCSI Name)
> > -- a university or other organization providing the component
> > -- a testing tool
> >
> > For the "Random" type, the naming authority field should be a random
> > or pseudo-random number. (See below on how this affects "conservative
> > reuse").
> >
> > [Note: the "type" field needn't be this big, but NDT felt that (a)
> > 2bits was insufficient for the future, (b) the "type" field should be
> > first, and (c) the "naming authority" field should be byte-aligned.]
> >
> > 2) Conservative Reuse
> >
> > The "conservative reuse" principle of this proposal just says that
> > SCSI Initiator Portnames should be viewed as static things (as much as
> > possible) and reused (when feasible) to give the most stable
> > presentation of SCSI Initiator Ports to both the OS above and to SCSU
> > targets across the wire.
> >
> > Whereas the current draft says "The ISID RULE does not preclude the
> > use of the same ISID to different Target portal groups", the
> > "conservative reuse" requirement would say that such reuse (using the
> > same ISID to many target portal groups) is a SHOULD (or is that
> > MUST?).  So, in one implementation, each iSCSI Initiator Portal group
> > would get configured with iSCSI Name, and a (small) set of ISIDs.  The
> > portal group might take this set of ISIDs as an ordered list.  The
> > first session to a Target portal group would use the first ISID, the
> > second to the *same* target portal group would use the second in the
> > list, etc.  The number of ISIDs would then define a configuration
> > parameter that can be interpreted as the maximum number of sessions
> > that can be built to a given target portal group.
> >
> > To facilitate "conservative reuse", the "qualifier" field of a set of
> > ISIDs SHOULD be generated using either a repeatable algorithm
> > (non-random or pseudo-random but based on a fixed seed) or any
> > algorithm and stored in a persistent location (e.g., registry or /etc
> > file).
> >
> > For the "Random" type, "conservative reuse" may not be an issue (say,
> > user application that doesn't care about reservations, etc.).  When it
> > is, the "naming authority" field SHOULD also be generated by a
> > mechanism similar to that for the "qualifier" field as specified above
> > (e.g., defined in the SW at compilation time.)
> >
> > DOCUMENTATING CHANGES:
> >
> > The iscsi drafts would need the following minor changes to support
> > this proposal:
> >
> > NDT doc
> > (a) define the structure of the ISID field (as described above)
> > (b) add some notes about implementation in the presense of
> > "conservative reuse" (e.g., that ISID should be configured once, say
> > at install time, then reused on subsequent reboots, even for "Random"
> > type).
> >
> > iSCSI MAIN doc:
> > (a) move the CID field in Login Request to another reserved field.
> > (b) expand the ISID field in Login Request and Response to encompass
> > the previous 4 bytes.
> > (c) add a sentence where the ISID field is defined indicating that
> > this field is a structured value, showing the structure (as above) and
> > point to the NDT document.
> > (d) add some text to "Note to Implementors" on "conservative reuse".
> >
> > Comments?
> >
> > Jim Hafner