Re: iscsi : changes involving tgt portal group tag.

["John Hufferd" <hufferd@us.ibm.com>]

Perhaps we need to discover what causes the TPGT to change.   I think you
are saying that the IP address has somehow changed to a different portal
group.  Since the SCSI Port is not suppose to change its "effective" name,
if one was to change the IP address, you have a serious implicit change in
effect.

I think that means that the Target Storage Controller has been taken
offline and reconfigured.  Now if you believe that across this probable
manual reconfiguration (including a probable power down), that the Target
will maintain this key state (like persistent reserves) I think we are
talking about a small corner case, which few would really expect.
Especially since there is usually a normal quiesce and the session stopped
normally when that type of thing is planed to be done.  Further, one could
argue that if a Target chose to keep the state, it had the obligation to
keep the same mapping of IP address to TPG.  (And the target would have to
have more then one Target Portal Group.)  If one was to say that the outage
was not planned, then usually what happens in this type of a situation, the
HW is fixed and restarted, not reconfigured.

In fact, lets understand where the Portal gets its IP address.  It gets it
from the configuration software that comes with the controller.  In other
words it is the configuration software (perhaps with Administrative
assistance) that assigns IP address to the portals that make up a portal
group.  It seems like if they assign it to begin with, then there is an
obligation to keep the IP address bound to the TPG as long as a persistent
state exist with an LU that has a Nexus through that Portal group (SCSI
Port).  IP addresses do not automatically move with the physical
HBA/Portals.

If I was to write an Initiator, and the above possibility worried me, I
think I would have a time value, which if passed, I would always rediscover
the Node.  That value can be anything, so if "defaultTime2Wait" is not
good, I think I would pick another time, which reflected the possibility of
a manual change.

Bottom line, I do not think we need to have TPGT in the Login.

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136, Cell: (408) 499-9702
Internet address: hufferd@us.ibm.com


"Mallikarjun C." <cbm@rose.hp.com> on 03/12/2002 04:09:33 PM

To:    John Hufferd/San Jose/IBM@IBMUS
cc:    <ips@ece.cmu.edu>, <t10@t10.org>
Subject:    Re: iscsi : changes involving tgt portal group tag.



John,

> I do not see the absolute requirement for this TPGT in the Login.

That was my initial inclination as well, but now I believe there's value
in adding the TPGT to the Login Request PDU.

> it would be approprate to say that if the time for the Reconnect has been
> more then the "...Time2Wait" that the Initiator, if it cares about the

If I understand you correctly, you are suggesting that the DefaultTimeWait
value negotiated *for a session* should be used for decisions after the
end of that session.  I am afraid that it's a slippery slope...

OTOH, regardless of the proposed change, it would be reasonable to add text
that generally expresses this idea of potential volatility of portal
<->portal group
association in the iSCSI (or perhaps NDT) draft.

On to reasons that convinced me....

- An iSCSI session is an I_T nexus, and it is between an initiator *port*
   and a target *port*.  Login as a mechanism to instantiate/add to an
   iSCSI session should identify the target port in question, not just the
   target node.  The initiator port is currently identified in the Login
   dialogue
  (the InitiatorName text key and the ISID field), but not the target port.

- When there has been a portal group change for an IP address (portal) -
   meaning its TPGT had changed - it would be much more quicker to
   identify the fact and prevent an I_T nexus establishment by way of
   the TPGT in Login Request PDU.  The other option of having each
   LU assert its UA (REPORTED LUNS DATA HAS CHANGED) on
    the first command is prone to errors (see next bullet), and simply
    ineffcient.

- A target cold reset or a powercycle (possibly done after the portal group
   reconfiguration) would clear the aforementioned UA (but would assert
   a different UA for the power-on reset, but this generic UA gives no
   clue to initiators on the need to re-discover the target ports).

- At the moment, it is unclear to me if SPC-3 is mandating an "interlocked"
   style UA for the REPORTED LUNS DATA HAS CHANGED condition
   (though it seems like it... I will raise it only on t10 reflector under
   a different
    cover).  If that isn't the case, the UA interlock capability
    (T10/00-359) would
    need to deployed as well to realibly deal with this portal group
    reconfiguration.

> That would address the issue with out any protocol changes.

It is certainly a PDU change, but one can argue (successfully, I think)
that
it is not a "protocol" change per se -  since the implicit usage of TPGT so
far
(see my message to Julian earlier on this thread) is merely being turned
into
an explicit usage.

To summarize, I realize the sensitivity of changes this late but I believe
there
are reasonable grounds here.

Regards.
--
Mallikarjun

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions Organization
Hewlett-Packard MS 5668
Roseville CA 95747
cbm@rose.hp.com

----- Original Message -----
From: "John Hufferd" <hufferd@us.ibm.com>
To: "Mallikarjun C." <cbm@rose.hp.com>
Cc: <ips@ece.cmu.edu>; <t10@t10.org>
Sent: Tuesday, March 12, 2002 11:07 AM
Subject: Re: iscsi : changes involving tgt portal group tag.


>
> Mallikarjun,
> I do not see the absolute requirement for this TPGT in the Login.  I
think
> it would be approprate to say that if the time for the Reconnect has been
> more then the "...Time2Wait" that the Initiator, if it cares about the
> TPGT,  SHOULD perform a rediscovery.
>
> That would address the issue with out any protocol changes.
>
> .
> .
> .
> John L. Hufferd
> Senior Technical Staff Member (STSM)
> IBM/SSG San Jose Ca
> Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
> Home Office (408) 997-6136, Cell: (408) 499-9702
> Internet address: hufferd@us.ibm.com
>
>
> "Mallikarjun C." <cbm@rose.hp.com>@ece.cmu.edu on 03/12/2002 10:30:29 AM
>
> Sent by:    owner-ips@ece.cmu.edu
>
>
> To:    "Shailesh Manjrekar" <shaileshm@aarohicommunications.com>
> cc:    <ips@ece.cmu.edu>, <t10@t10.org>
> Subject:    Re: iscsi : changes involving tgt portal group tag.
>
>
>
> Shailesh,
>
> The failure resulting out of a TPGT mismatch (assuming we have the TPGT
in
> the Login Request PDU), would have to trigger a discovery
> (SendTargets/SLP/...)
> updating the initiator of the latest configuration.  This discovery
process
> is
> similar
> to the FCP ADISC process you refer to.
>
> Alternatively, if there has been a change in the LU view of  a given
target
> portal
> group tag (meaning that the TPGT would match correctly), the LUs in the
> view
> should have established UA with an ASC of REPORTED LUNS DATA HAS
> CHANGED since the SCSI port association has changed for the LUs.  This
> again
> should trigger a discovery process from the initiator.
>
> It seems to be that we are now sufficiently covered either way.
> --
> Mallikarjun
>
> Mallikarjun Chadalapaka
> Networked Storage Architecture
> Network Storage Solutions Organization
> Hewlett-Packard MS 5668
> Roseville CA 95747
> cbm@rose.hp.com
>
>
> ----- Original Message -----
> From: "Shailesh Manjrekar" <shaileshm@aarohicommunications.com>
> To: "'Mallikarjun C.'" <cbm@rose.hp.com>; "'Julian Satran'"
> <Julian_Satran@il.ibm.com>
> Cc: <ips@ece.cmu.edu>; <t10@t10.org>
> Sent: Monday, March 11, 2002 7:51 PM
> Subject: RE: iscsi : changes involving tgt portal group tag.
>
>
> > Hi,
> >
> > On a TPG reconfiguration or TPGT reassignment, shouldn't there be a
> > discovery followed by SCSI Inquiry and Report_LUNS which would make the
> > Initiator aware of this change? Can the  Async message - iscsi Event
> > Data notify the Initiator of the change, which would force an
discovery.
> > This would be similar to an ADISC for FCP. Because including the TPGT
in
> > the login would prevent inadvertent logins but would still not notify
> > the initiator of the changed configuration?
> >
> > Thanks,
> > Shailesh.
> > Aarohi Communications.
> > -----Original Message-----
> > From: owner-t10@t10.org [mailto:owner-t10@t10.org] On Behalf Of
> > Mallikarjun C.
> > Sent: Wednesday, March 06, 2002 10:17 AM
> > To: Julian Satran
> > Cc: ips@ece.cmu.edu; t10@t10.org
> > Subject: Re: iscsi : changes involving tgt portal group tag.
> >
> > * From the T10 Reflector (t10@t10.org), posted by:
> > * "Mallikarjun C." <cbm@rose.hp.com>
> > *
> > > The issue is that I am not sure that a target is checking today
> > anything -
> > > (adapter drivers may check oif in their realm they can give out a
> > TSID).
> > > Nothing else is needed.
> >
> > Target is required to derive the TPGT and use it in both cases of Login
> > -
> >      a) when TSID = 0, to ascertain if a session reinstatement needs to
> >          be effected for that TPGT.
> >      b) when TSID != 0, to ascertain the validity of TSID and add in
> >           the new connection to an existing session if it is valid for
> > that TPGT.
> > --
> > Mallikarjun
> >
> > Mallikarjun Chadalapaka
> > Networked Storage Architecture
> > Network Storage Solutions Organization
> > Hewlett-Packard MS 5668
> > Roseville CA 95747
> >
> > ----- Original Message -----
> > From: "Julian Satran" <Julian_Satran@il.ibm.com>
> > To: "Mallikarjun C." <cbm@rose.hp.com>
> > Cc: <ips@ece.cmu.edu>; <t10@t10.org>
> > Sent: Tuesday, March 05, 2002 11:56 PM
> > Subject: Re: iscsi : changes involving tgt portal group tag.
> >
> >
> > > The issue is that I am not sure that a target is checking today
> > anything -
> > > (adapter drivers may check oif in their realm they can give out a
> > TSID).
> > > Nothing else is needed. Does SCSI have to be aware of it? Perhaps but
> > > iSCSI certainly not. Does a "front-end" have to be - again probably
> > not
> > > the name identifies the node.
> > >
> > > Julo
> > >
> > >
> > >
> > >
> > > "Mallikarjun C." <cbm@rose.hp.com>
> > > 05-03-02 22:34
> > > Please respond to "Mallikarjun C."
> > >
> > >
> > >         To:     Julian Satran/Haifa/IBM@IBMIL
> > >         cc:     <ips@ece.cmu.edu>, <t10@t10.org>
> > >         Subject:        Re: iscsi : changes involving tgt portal
group
> > tag.
> > >
> > >
> > >
> > > Julian,
> > >
> > > Whatever methods a target is expected to use today to derive the
> > implicit
> > > TPGT to preclude a duplicate I-T nexus would work after the change as
> > > well,
> > > except the derived value needs to be compared against the (proposed)
> > > TPGT in the Login Request payload.
> > >
> > > Please comment if we're missing something.
> > >
> > > Regards.
> > > --
> > > Mallikarjun
> > >
> > > Mallikarjun Chadalapaka
> > > Networked Storage Architecture
> > > Network Storage Solutions Organization
> > > Hewlett-Packard MS 5668
> > > Roseville CA 95747
> > >
> > > ----- Original Message -----
> > > From: "Julian Satran" <Julian_Satran@il.ibm.com>
> > > To: "Mallikarjun C." <cbm@rose.hp.com>
> > > Cc: <ips@ece.cmu.edu>; <owner-ips@ece.cmu.edu>; "Santosh Rao"
> > > <santoshr@cup.hp.com>;
> > > <t10@t10.org>
> > > Sent: Monday, March 04, 2002 10:24 PM
> > > Subject: Re: iscsi : changes involving tgt portal group tag.
> > >
> > >
> > > > Has a decent target implementation and easy way of checking that
the
> > TPG
> > > > is correct?
> > > >
> > > > Julo
> > > >
> > > >
> > > >
> > > >
> > > > "Mallikarjun C." <cbm@rose.hp.com>
> > > > Sent by: owner-ips@ece.cmu.edu
> > > > 05-03-02 01:12
> > > > Please respond to "Mallikarjun C."
> > > >
> > > >
> > > >         To:     "Santosh Rao" <santoshr@cup.hp.com>,
> > <ips@ece.cmu.edu>
> > > >         cc:     <t10@t10.org>
> > > >         Subject:        Re: iscsi : changes involving tgt portal
> > group
> > > tag.
> > > >
> > > >
> > > >
> > > > Santosh and Jim,
> > > >
> > > > It appears a good idea to add in the portal group tag in the Login
> > > > Request header for a sanity check by the receiving target portal.
> > > >
> > > > I generally agree with Jim's comments that the duration of
> > persistence
> > > > of a portal group tag is intricately linked to the extent of portal
> > > > reconfiguration.
> > > > Not every target reconfiguration may result in a portal group tag
> > > > reassignment.
> > > > OTOH, some reconfigurations may be so extensive as to cause even a
> > node
> > > > name change.
> > > >
> > > > Some comments on Santosh's message -
> > > >
> > > > > "If a portal group is re-configured such that all its previously
> > > > > advertised network portals are no longer a part of the portal
> > group,
> > > > > then, the portal group tag (and thereby, the port
name/identifier)
> > > > > *MUST* be changed to indicate a new target port."
> > > >
> > > > I am not sure this solves the problem you're trying to get at.  If
> > none
> > > of
> > > > the earlier IP addresses can get an initiator to the SCSI target
> > port
> > > that
> > > > it
> > > > knew of (your scenario), it appears to me that it doesn't matter if
> > the
> > > > portal group tags are changed or not....A new discovery process
> > should
> > > > update the initiator of the changed portal addresses.
> > > >
> > > > I suggest the following text -
> > > >
> > > >    After a portal group reconfiguration which changed the view of
> > LUs
> > > >    for an initiator with a given set of privileges, the target MUST
> > > change
> > > >    the portal group tag that represents the reconfigured target
> > portal
> > > > group.
> > > >
> > > > > > Under these events, shouldn't all "open/active I_T_L traffic"
> > have
> > > > been
> > > > > > aborted, closed or otherwise ended?  So this shouldn't be an
> > issue
> > > at
> > > > all.
> > > > >
> > > > > On a session logout & re-login, it is not efficient/necessary to
> > close
> > > > > and re-open each LUN behind that target port, due to the fact
that
> > a
> > > > > target port may have hundred's of LUNs behind it.
> > > >
> > > > I agree with Jim on this one - there should be *no* open/active
> > I_T_L
> > > > nexus
> > > > traffic after a reconfiguration, targets can enforce this via
simple
> >
> > > iSCSI
> > > > means
> > > > (reject initiator advances to continue the session for
> > DefaultTime2Wait+
> > > > DefaultTime2Retain seconds).  In fact, Async logout request
requires
> > a
> > > > clean closure that implicitly aborts I/Os.
> > > >
> > > > What you're describing is typical O/S "LUN open" and "LUN close"
> > > > interactions.  I agree that they wouldn't have to be repeated, but
> > care
> > > > must be taken to ensure that new I/Os (on the new session after
> > > > reconfiguration)
> > > > are not delivered to the incorrect LUs.  It seems that the addition
> > of
> > > > TPGT in the login header and the proposed new text (above) would
> > take
> > > > care of this.
> > > >
> > > > Regards.
> > > > --
> > > > Mallikarjun
> > > >
> > > > Mallikarjun Chadalapaka
> > > > Networked Storage Architecture
> > > > Network Storage Solutions Organization
> > > > Hewlett-Packard MS 5668
> > > > Roseville CA 95747
> > > >
> > > > ----- Original Message -----
> > > > From: "Santosh Rao" <santoshr@cup.hp.com>
> > > > To: <ips@ece.cmu.edu>
> > > > Cc: <t10@t10.org>
> > > > Sent: Monday, March 04, 2002 10:40 AM
> > > > Subject: Re: iscsi : changes involving tgt portal group tag.
> > > >
> > > >
> > > > > * From the T10 Reflector (t10@t10.org), posted by:
> > > > > * Santosh Rao <santoshr@cup.hp.com>
> > > > > *
> > > > > Jim,
> > > > >
> > > > > I agree that a "complete re-configuration" of a target port can
> > result
> > > > > in a new port name & port identifier. However, the tricky part is
> > the
> > > > > definition of a "complete re-configuration of an iscsi target
> > port",
> > > due
> > > > > to the concepts of portal groups involving multiple network
> > portals.
> > > > >
> > > > > For example, if a portal group (aka, an iscsi target port) were
to
> > be
> > > > > re-configured to include a new network portal [moved from another
> > > portal
> > > > > group], then, the target port itself has not changed, since it is
> > > still
> > > > > accessible through its previously used network portals. What has
> > > changed
> > > > > is the individual network portal that has moved from 1 target
port
> > to
> > > > > another.
> > > > >
> > > > > Hence, it is sufficient, in this case, to maintain persistence of
> > the
> > > > > target port name/identifier, without requiring any change in port
> > > > > name/identifier. By requiring initiators to send the intended
TPGT
> >
> > > (scsi
> > > > > target port name/identifier) along with the login request, this
> > allows
> > > > > the target port to detect that the network portal is being
> > accessed to
> > > > > connect to a different target port and it can reject the login.
> > > > >
> > > > > It may be helpful to call out the specific case when a port
> > > > > name/identifier MUST change. How about something like :
> > > > >
> > > > > "If a portal group is re-configured such that all its previously
> > > > > advertised network portals are no longer a part of the portal
> > group,
> > > > > then, the portal group tag (and thereby, the port
name/identifier)
> > > > > *MUST* be changed to indicate a new target port."
> > > > >
> > > > > This would allow access to the target port through its un-altered
> > > > > network portals to continue un-disrupted. More comments inline,
in
> > > > > response to some of your queries.
> > > > >
> > > > > Thanks,
> > > > > Santosh
> > > > >
> > > > > NOTE : In this discussion target port and target portal group are
> > used
> > > > > to imply the same entity, within a target node.
> > > > >
> > > > >
> > > > > Jim Hafner wrote:
> > > > >
> > > > > > SAM-2 requires scsi port names to be persistent and
> > > world-wide-unique.
> > > > > > (SAM-2 Rev 22 Section 4.7.7). Quoting from SAM-2 :
> > > > > >
> > > > > > "A scsi port name shall never change and may be used to
> > persistently
> > > > > > identify a scsi initiator port or target port...".
> > > > > >
> > > > > > <JLH>
> > > > > > There are different ways that one can interpret this
> > "persistent"
> > > > rule. The
> > > > > > intent was that names shouldn't change over time when
> > *identifying
> > > the
> > > > same
> > > > > > object*.   What that means is that if the object changes (in
any
> > > > > > discernable way), then the name should change.  So, the object
> > can
> > > > move to
> > > > > > a different piece of hardware, but it can still be named the
> > same
> > > way.
> > > >  If
> > > > > > the object changes, e.g., in this case, reconfigures as a
> > different
> > > > set of
> > > > > > network portals with different addressing elements, then I
would
> >
> > > think
> > > > that
> > > > > > the name should change as well.   That's all the persistence
one
> > > > really
> > > > > > needs.
> > > > > >
> > > > > > I'm not sure what that implies about your recommendation.  I
> > don't
> > > see
> > > > any
> > > > > > problem with it, either.
> > > > > > </JLH>
> > > > >
> > > > > I think we may be in agreement. (See reasoning above).
> > > > >
> > > > >
> > > > > >
> > > > > > The rationale for (2) is :
> > > > > > --------------------------
> > > > > > Consider an initiator node establishing multiple sessions to a
> > scsi
> > > > tgt
> > > > > > port, with each session established to a subset of the network
> > > portals
> > > > > > within the tgt portal group.
> > > > > >
> > > > > > Consider an iscsi transport event involving the
re-configuration
> > of
> > > > > > target portal groups on the iscsi target node. This may be
> > preceeded
> > > > by
> > > > > > the iscsi sessions seeing an async message "target requests
> > logout",
> > > > > > followed by session logout, portal group re-configuration, and
> > then,
> > > > the
> > > > > > initiator re-establishes sessions.
> > > > > >
> > > > > > Across a transport event that results in such session
> > termination
> > > and
> > > > > > re-establishment, the initiator needs to authenticate that it
is
> >
> > > still
> > > > > > speaking to the same [i]scsi target port, in order to ensure
> > that
> > > any
> > > > > > open/active I-T-L nexus traffic on that session is not
> > incorrectly
> > > > > > routed to a wrong LUN after such a transport event.
> > > > >
> > > > > > <JLH>
> > > > > > Under these events, shouldn't all "open/active I_T_L traffic"
> > have
> > > > been
> > > > > > aborted, closed or otherwise ended?  So this shouldn't be an
> > issue
> > > at
> > > > all.
> > > > >
> > > > > On a session logout & re-login, it is not efficient/necessary to
> > close
> > > > > and re-open each LUN behind that target port, due to the fact
that
> > a
> > > > > target port may have hundred's of LUNs behind it.
> > > > >
> > > > > All outstanding I/Os need to be aborted. Once the session is
> > > > > re-established [and the target port is authenticated to be the
> > same],
> > > > > further I/O traffic can be resumed without requiring the SCSI ULP
> > to
> > > > > close and re-open each LUN. Some transport specific clearing
> > effects
> > > may
> > > > > have occurred, which may require additional LUN level activity,
in
> >
> > > some
> > > > > cases.
> > > > >
> > > > > (There are analogies to the above model in FCP & SRP, which
> > > authenticate
> > > > > port name/identifier on login, allowing I/O resumption after
> > session
> > > > > termination and re-establishment.)
> > > > >
> > > > >
> > > > > > To prevent such authentication issues, iscsi can send the iscsi
> > > target
> > > > > > port identifier (portal group tag) explicitly in the login
> > request,
> > > > and
> > > > > > the login can be rejected by the target on a portal group tag
> > > > mis-match.
> > > > > > (if the network portal does not belong to the addressed portal
> > group
> > > > > > tag).
> > > > > > <JLH>
> > > > > > Did you mean for the initiator to send this TPGT?
> > > > > > </JLH>
> > > > >
> > > > > Yes. The initiator login request must include the target portal
> > group
> > > > > tag, thus identifying the target port to which a session is being
> > > > > established.
> > > > >
> > > > > Login currently carries no addressing information, since the
> > > addressing
> > > > > is implicit, based on the TCP connection in use. The problem with
> > this
> > > > > approach is that the login implicitly addresses a network portal,
> > and
> > > > > not the target port. As seen in the earlier example, the
> > association
> > > of
> > > > > network portal <-> target port can change, and such changes can
be
> > > > > detected, if the initiator were to explicitly identify the target
> > port
> > > > > being logged into.
> > > > >
> > > > > --
> > > > > ##################################
> > > > > Santosh Rao
> > > > > Software Design Engineer,
> > > > > HP-UX iSCSI Driver Team,
> > > > > Hewlett Packard, Cupertino.
> > > > > email : santoshr@cup.hp.com
> > > > > Phone : 408-447-3751
> > > > > ##################################
> > > > > *
> > > > > * For T10 Reflector information, send a message with
> > > > > * 'info t10' (no quotes) in the message body to majordomo@t10.org
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> > *
> > * For T10 Reflector information, send a message with
> > * 'info t10' (no quotes) in the message body to majordomo@t10.org
> >
> >
>
>
>
>
>