Re: iSCSI: ERL=1 question.

["Mallikarjun C." <cbm@rose.hp.com>]

Just one minor change in the text.

In sub-bullet (ii), replace "Data-SNACK reject" with
"SNACK rejected". 

The former is a Reject reason code, and the latter 
is an iSCSI response reason with a defined key-ASC-ASCQ
in section 3.4.3.
-- 
Mallikarjun 


Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions Organization
MS 5668	Hewlett-Packard, Roseville.
cbm@rose.hp.com


"Mallikarjun C." wrote:
> 
> Prasenjit,
> 
> I tend to agree with you.
> 
> The "must send status after SNACK reject" rule you pointed out was
> originally introduced to avoid infinite loops where an initiator
> continually asks for data re-transmit, and target keeps rebuffing
> (for transient internal reasons, since data SNACK is legal for
> ErrorRecoveryLevel=1).  The intent was to force the I/O completion,
> precluding SNACK retries.
> 
> In the case you describe, the target already delivered the status, so
> I propose that all we need to ensure is that there are no SNACK retries.
> Initiator should be able to internally assume a "Data-SNACK reject"
> reason - it does so today anyway if no recovery for the lost data is
> attempted.
> 
> Here is the new text for consideration (to replace the bullet (a) text
> on page 134 in section 8.4 of rev09) -
> 
> a) Request the desired data PDU through SNACK. In its turn,
>    the target MUST either resend the data PDU or, reject the
>    SNACK with a Reject PDU with a reason-code of "Data-SNACK
>    Reject" in which case
>         (i) if the status had not already been sent for the
>             command, the target MUST terminate the command with
>             an iSCSI command response reason of "SNACK rejected".
>         (ii)if the status was already sent, no further action is
>             necessary for the target.  Initiator in this case
>             MUST internally signal the completion with the "Data-SNACK
>             reject" reason disregarding any received status PDU,
>             but must wait for the status to be received before doing
>             so.
>    [OR].....
> 
> Note that there's one paranoid case here in sub-bullet (ii) above
> where a status SNACK (if status is also lost...) will cause the target
> to report the original status again (SNACK demands a replica) with a
> successful completion.  But in reality, the intervening "Data-SNACK
> Reject" had already sealed the fate of the I/O on the initiator.
> The retransmitted status merely fills the StatSN hole and gives the
> initiator the determinism to re-use the ITT.
> 
> The wording in sub-bullet (ii) will cover this paranoid case as well.
> 
> Comments?
> 
> To Santosh:
> 
> > > upon
> > > encountering a datasn hole, say, data_snack_sent, and it should ignore
> > > any scsi status received while in this state.
> 
> I wouldn't recommend that!  If the Data SNACK were successful, this
> status
> is the final status for the I/O.  Discarding the status would mean
> setting
> the stage for a(n unnecessary) status SNACK in future.
> --
> Mallikarjun
> 
> Mallikarjun Chadalapaka
> Networked Storage Architecture
> Network Storage Solutions Organization
> MS 5668 Hewlett-Packard, Roseville.
> cbm@rose.hp.com
> 
> Prasenjit Sarkar wrote:
> >
> > Santosh,
> >
> > Regarding (B), I did not even venture to imply that there would be a
> > stat_sn hole.
> >
> > I'm a little confused about your data snack recovery rule, but that is
> > besides the point and I can discuss this offline with you.
> >
> > My chief concern is that ERL 1 leaves open the possibility for duplicate
> > SCSI status PDUs. There are three solutions:
> >
> > 1. Leave it to implementors who can use their judgement to decide which
> > status to ignore.
> > 2. Formulate a tight rule: status PDU with higher stat_sn number always
> > wins.
> > 3. Change the standard so that duplication does not happen.
> >
> > I would like the standard to at least discuss this and then state what path
> > they chose and why (with good reason of course). I do not think that this
> > is merely an implementation issue.
> >
> > My preference is of course: 3,2,1.
> >
> >    Prasenjit Sarkar
> >    Research Staff Member
> >    IBM Almaden Research
> >    San Jose
> >
> >
> >                     Santosh Rao
> >                     <santoshr@cup.       To:     Prasenjit Sarkar/Almaden/IBM@IBMUS
> >                     hp.com>              cc:     ips@ece.cmu.edu
> >                     Sent by:             Subject:     Re: iSCSI: ERL=1 question.
> >                     owner-ips@ece.
> >                     cmu.edu
> >
> >
> >                     01/03/2002
> >                     06:45 PM
> >
> >
> >
> > Prasenjit,
> >
> > Regarding your question (B)....
> >
> > Anytime the initiator's receive path successfully receives a pdu and
> > passes format and digest error tests, the local state information of
> > exp_cmdsn, max_cmdsn, statsn should be updated based on the values that
> > are carried in that pdu and the rules specified in Section 2.2.2.
> >
> > Thus, in your below context, when the first scsi status pdu was
> > received, the exp_statsn should have gotten updated and there should not
> > have been any hole in statsn sequence. What am I missing (?)
> >
> > Regarding your question (A)...
> >
> > You may have read more into my answer than was intended.
> >
> > It is not the job of the lower layered pdu assembly path to filter out
> > duplicate scsi status pdu's. Rather, this intelligence would lie within
> > the module (finite state machine) that implemented the i/o path logic.
> >
> > While a data_snack has been sent and the i/o state machine is awaiting
> > missing data pdu's , any received scsi status for that exchange must not
> > be sent up to the SCSI ULP. Upon successful receipt of all the missing
> > data pdu[s], the status can then be conveyed to the ULP.
> >
> > (I don't have a strong opinion either way, if we should have to state
> > the above in the draft, or leave it unsaid as implementation specifics.
> > I guess, at some point we cross over into the realm of an implementation
> > guide vs a protocol specification :)
> >
> > - Santosh
> >
> > Prasenjit Sarkar wrote:
> > >
> > > I'm assuming Santosh wanted to reply to the entire group, so I am
> > > forwarding his response to the mailing list.
> > >
> > > Santosh's argument is to provide the (unstated) rule that data assembly
> > is
> > > the domain of a lower iSCSI layer whose job is to filter out duplicate
> > SCSI
> > > status PDUs. I would gladly accept this argument except that a standard
> > > should present a generic rule which should be independent of how a
> > protocol
> > > is implemented. I would prefer a rule based on stat_sn ordering or any
> > > other equivalent rather than one based on ideal iSCSI layering.
> > >
> > > Santosh's motivation of having a seamless way of completing commands is
> > > probably not true as initiators are allowed to generate SCSI responses.
> > >
> > >    Prasenjit Sarkar
> > >    Research Staff Member
> > >    IBM Almaden Research
> > >    San Jose
> > >
> > >
> > >                     Santosh Rao
> > >                     <santoshr@cup.       To:     Prasenjit
> > Sarkar/Almaden/IBM@IBMUS
> > >                     hp.com>              cc:
> > >                     Sent by:             Subject:     Re: iSCSI: ERL=1
> > question.
> > >                     santoshr@cup.h
> > >                     p.com
> > >
> > >
> > >                     01/03/2002
> > >                     05:00 PM
> > >
> > >
> > >
> > > Prasenjit,
> > >
> > > Responses inline.
> > >
> > > Regards,
> > > Santosh
> > >
> > > Prasenjit Sarkar wrote:
> > > >
> > > > Assume the following scenario where I and T stand for initiator and
> > > target
> > > > respectively.
> > > >
> > > > 1. I->T: Scsi Cmd
> > > >
> > > > 2. T->I: Scsi Data (DataSN:0)
> > > >
> > > > 3. T->I: Scsi Status (Good)
> > > >
> > > > Assume there is a data digest problem for the data with DataSN:0, so
> > > >
> > > > 4. I->T: Data Snack for DataSN:0
> > > >
> > > > The target for some reason cannot respond with the data, so according
> > to
> > > > the spec
> > > >
> > > > 5: T->I: Reject with reason DATA_SNACK_REJECT
> > > >
> > > > 6. T->I: Scsi Status (iSCSI response: SNACK rejected -> SCSI READ
> > Error)
> > > >
> > > > The questions are as follows:
> > > >
> > > > A. Is SAM ambivalent of the fact that there can be two statii for the
> > > same
> > > > command? (I dont have a problem if SAM doesnt)
> > >
> > > I'm not certain on what SAM says on this one. However, one can envision
> > > that the iscsi i/o finite state machine would enter into some state upon
> > > encountering a datasn hole, say, data_snack_sent, and it should ignore
> > > any scsi status received while in this state.
> > > The lower pdu receive and re-assembly module should have passed format
> > > and digest checks (assuming the frame came thru fine) and so, the local
> > > value of exp_statsn should have gotten updated within this initiator
> > > instance's iscsi connection finite state machine.
> > >
> > > Thus.....
> > > i) since the 1st status was ignored while awaiting a data_snack
> > > response, only 1 scsi status ends up being processed by the i/o finite
> > > state m/c.
> > >
> > > ii) there will not be a statsn hole, since the previous status was
> > > received successfully by the receive path, resulting in an update to
> > > exp_statsn. however, the staus was discarded within the i/o fsm.
> > >
> > > >
> > > > B, Does the second SCSI status have the same StatSN as the first?
> > Likely,
> > > > it does not, but it should be clearly stated that a SCSI status with
> > > higher
> > > > stat_sn overrides one with the lower stat_sn.
> > >
> > > See above.
> > >
> > > >
> > > > C. I'm looking for motivation here: why does the target (rather than
> > the
> > > > initiator) generate the second status? Couldnt the initiator also do
> > the
> > > > same on receiving the DATA_SNACK_REJECT?
> > >
> > > So that all I/Os receive a single method of completion via the SCSI
> > > status, I guess.
> > >
> > > --
> > > ##################################
> > > Santosh Rao
> > > Software Design Engineer,
> > > HP-UX iSCSI Driver Team,
> > > Hewlett Packard, Cupertino.
> > > email : santoshr@cup.hp.com
> > > Phone : 408-447-3751
> > > ##################################
> >
> > --
> > ##################################
> > Santosh Rao
> > Software Design Engineer,
> > HP-UX iSCSI Driver Team,
> > Hewlett Packard, Cupertino.
> > email : santoshr@cup.hp.com
> > Phone : 408-447-3751
> > ##################################