Re: Fw: iSCSI: changing MaxPDUDataLength

["Julian Satran" <Julian_Satran@il.ibm.com>]

Mallikarjun pointed that to me - so - using offset in SNACK is not a practical proposition because the offset is not known.
The initiator misses something - it does not know what; it knows only that it missed a number.

keeping track of any byte sent (scoreboarding) is not something easily doable in SCSI - where the target is the "transfer master".
I am a bit obscure but I assume that Eddy understands already.

Julo

"Mallikarjun C." <cbm@rose.hp.com> 06/13/2002 09:57 PM Please respond to "Mallikarjun C."

To:        Julian Satran/Haifa/IBM@IBMIL         cc:                 Subject:        Fw: iSCSI: changing MaxPDUDataLength

----- Original Message ----- 
From: "Eddy Quicksall" <eddy_quicksall@ivivity.com>
To: "Mallikarjun C." <cbm@rose.hp.com>
Sent: Thursday, June 13, 2002 4:51 AM
Subject: RE: iSCSI: changing MaxPDUDataLength


> I thought I always said that data-in was the issue. If I didn't make that
> clear, I apologize.
> 
> If there is no reason other than making a point and keeping consistency, can
> you please talk over the likelihood of using an offset and data length
> instead of BegRun and RunLength? If that were done, the target would not
> have any issues to deal with.
> 
> 
> >All you need on a per-task basis is really the value of one DataSN
> >*where* it had changed.
> 
> Yes, this is one thing I've been driving at. If hardware is independently
> handling the transmission, it is likely in a loop sending PDU's with
> MaxPDUDataSegmentLength. The firmware would have to pause the hardware; then
> go to each TCB and record the current DataSN; then inform the hardware of
> the new Length; then unpause the hardware. On most pieces of hardware I've
> seen in the past with a pause feature, there has always been some sort of
> oversight and a lot of ugly work around code had to be written.
> 
> If there are different processors used (one for the fast path and one for
> the slow path), then coordination between the processors has to take place
> also.
> 
> In hardware, it is not uncommon to keep the TCBs in very fast memory. That
> type of design usually has a limit on memory because it is usually on chip.
> Now, to ask for an extra DWORD in every TCB for something that is 99% of the
> time unused is almost unheard of. If that DWORD were in external memory then
> there would likely have to be code that touches that memory each time a TCB
> is started (or finished).
> 
> You're pseudo code below has a slight flaw. Since, in a single TCB, some
> PDU's may have been transmitted with the old size and some with the new size
> and then a BegRun is some number of PDU's into the new size, you have to
> also use the new size and number of PDU's into the new size in your
> calculation. Not difficult code but just adds to the messiness that I would
> like to avoid.
> 
> I've mentioned four things here that account for why I said it is messy. I
> never tried to say its not doable or conceptually simple.
> 
> Can you please consider using an offset?
> 
> 
> 
> Eddy
> 
> -----Original Message-----
> From: Mallikarjun C. [  <mailto:cbm@rose.hp.com> mailto:cbm@rose.hp.com]
> Sent: Wednesday, June 12, 2002 9:03 PM
> To: Eddy Quicksall
> Subject: Re: iSCSI: changing MaxPDUDataLength
> 
> 
> Eddy,
> 
> Comments inline, I think it's best to take it off the list (at least for a
> while).
> 
> Caveat: I am *still* not clear on the specifics you have in mind.  You
> talked
> about quiescing the commands, then data, then only read commands....
> 
> I have to apologize, but I am afraid I can't spend my cycles any further on
> this thread.
> --
> Mallikarjun
> 
> Mallikarjun Chadalapaka
> Networked Storage Architecture
> Network Storage Solutions
> Hewlett-Packard MS 5668
> Roseville CA 95747
> cbm@rose.hp.com
> 
> 
> ----- Original Message -----
> From: "Eddy Quicksall" <eddy_quicksall@ivivity.com>
> To: "Mallikarjun C." <cbm@rose.hp.com>
> Cc: <ips@ece.cmu.edu>
> Sent: Wednesday, June 12, 2002 2:19 PM
> Subject: RE: iSCSI: changing MaxPDUDataLength
> 
> 
> > Mallikarjun,
> >
> > This is not a case of arguing ... we are simply trying to understand each
> > others point over EMAIL, which is difficult at best. Please try to work
> with
> > me instead of against me.
> 
> I am fairly oblivious to the person arguing an issue, it's the issue and its
> consequences that matter to me.
> 
> >I could be wrong but I just need a little more
> > cooperation instead of using the overworked terms like "if you will just
> > read ...".
> 
> Which I didn't say...
> 
> >
> > You went into enough detail in one EMAIL to show that it is a messy job.
> > When a solution is too messy and hard to explain, it is usually a clue
> that
> > something needs to be simplified ... and that is what I am after.
> >
> > So far, I have not been able to catch the reason why we could not simply
> > specify that the initiator must idle the commands before issuing a new
> size.
> > Julian hinted that it would be a performance hit but I don't believe that
> > will be a performance hit because it should be rare. Please explain why it
> > would be a performance hit.
> 
> I can't explain what others said, but my point is that changing the PDU size
> in the middle of the I/O *does not have to* cause the explosion of metadata
> that you were fearing.
> 
> >
> > I would also like to know why the SNACK just doesn't simply ask for an
> > offset and a data length because that would simplify everything. Could you
> > please explain that?
> 
> I think to make the point that data retransmission requests can only be made
> at the PDU level, and also to be consistent with other types of SNACK
> (status,
> data ACK).
> 
> >
> > You mention data-out below but I'm not talking about data-out, I'm talking
> > about data-in.
> 
> But they may be related!  Here's what I said earlier -
>  "a) reads and writes may both be active in the typical case on an iSCSI
> connection,"
> I thought you were coming up with a rule that affects initiator iSCSI
> layer's ability to quickly
> react to PMTU changes.  If so, it would impact both reads and writes.
> 
> >
> > BTW, I don't have much background in ULPDU so maybe that is the key as to
> > why the initiator can't idle first. Can you please explain that?
> >
> 
> ULPDU means "ULP PDU" - ULP from the perspective of the framing layer.
> Please refer TUF draft for details.
> 
> >
> > I don't know what this has to do with a long write. Can you please explain
> > that?
> 
> I already explained this.....
>  " As I earlier hinted twice, the Data-Out PDUs will not be ULPDU-contained
>   anymore - and that could mean target would need to spend more memory/bus
>   bandwidth/computation to deal with those till the long write is done. "
> 
> > In re-reading your statement below, I am wondering if you understand the
> > problem ...
> 
> Yes, I do.
> 
> >The problem is that when a SNACK is sent and the PDU sizes have
> > changed due to MaxPDUDataSegmentLength, then it puts a burden on the
> target
> > to compute the proper offset of the BegRun (a.k.a messy).
> >
> > This is solved by the target in at least two ways:
> >
> > 1) The target can record the last PDU size when the change takes place.
> But
> > it would also have to keep track of the number of already completed PDU's
> > per outstanding command. This is because some commands may have missing
> > PDU's with the old size and others may have missing PDU's with the new
> size.
> > To make matters worse, some commands could have n PDUs already sent and
> > others could have m PDUs already sent. If you want, I could make up a
> > diagram of this and send it to you.
> 
> Not necessary.  I assume you're referring to read commands.  I already
> answered this - "Given that the changed PDU size is not specific to one
> command, I see the history of "past max PDU size(s)" as a connection
> attribute.  All you need on a per-task basis is really the value of one
> DataSN
> *where* it had changed." [ Assume that this is a TCB element called
> "PDU_size_changed_at". ]
> 
> Thinking about all permutations could be distracting.  All you want is
> "given a
> DataSN, translate it to a data offset without maintaining DataSN->PDU_size
> records for each past PDU you had earlier sent".  That should be quite
> doable
> with what I described
> 
> if (TCB.PDU_size_changed_at is invalid) {
>              Offset = SNACK.BegRun*Connection.PDU_size;
> } else {
>              Compute the offset based on whether BegRun is before or after
>              TCB.PDU_size_changed_at.  Use Connection.Previous_PDU_size.
> }
> 
>  The "if" condition computes to "true" 99% of the time.
> 
> This is as explicit as I can be.....
> 
> >
> > 2) The target could "force an idle of data-in commands" (by queuing them)
> > before it responds to the change request. Doing this would probably be no
> > different than the initiator doing it but it would be easier and less
> > intrusive for the initiator to do it.
> >
> >
> > Eddy
> 
> 
> 
>