SORT BY:

LIST ORDER
THREAD
AUTHOR
SUBJECT


SEARCH

IPS HOME


    [Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

    RE: ISCSI: Unsolicited data in draft v12



    
    Julian suggested:
    
    unsolicited data - if not immediate or none must be of FirstBufSize or Total
    Data (I am not sure we should allow immediate but I see no real harm). 
    
    Using what Julian suggested, can we change the wording to:
    
    "An iSCSI initiator MAY choose to send no unsolicited data,
    only immediate data or FirstBurstSize bytes of unsolicited data
    with a command. If any non-immediate unsolicited data is sent, 
    the total unsolicited data MUST be either the negotiated amount 
    or all the data if the total amount is less 
    than the negotiated amount for unsolicited data."
    
    Eddy
    
    -----Original Message-----
    From: John Hufferd [mailto:hufferd@us.ibm.com]
    Sent: Wednesday, April 10, 2002 2:42 PM
    To: Rod Harrison
    Cc: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2); 'Julian Satran';
    ips@ece.cmu.edu; owner-ips@ece.cmu.edu
    Subject: RE: ISCSI: Unsolicited data in draft v12
    
    
    
    OK, now we are on the same page.
    
    Though the example you state, would support your case, it seems really
    strange for someone using an HBA to have a MaxRecvPDULength greater then
    the FirstBurstSize.   I say that simply because one would normally think
    that it is the HBA that has the smaller amount of storage, hence a
    smaller
    MaxRecvPDULength.  However, I can accept that a target might have a very
    small unsolicited buffer space, but can permit very large Solicited
    Buffers, and might have an HBA, which has a large buffer on the card, so
    your example may be possible even with HBAs.
    
    You say that you have seen your example at plugfest, and I think that
    may
    be the case because most of the Plugfest implementations have been
    Software Targets, and in those cases, MaxRecevPDULength has nothing to
    do
    with the HBA (because there is none), so I think you point may be valid,
    and is perhaps reasonable in those cases.
    
    So even though we had to drift  quite a ways,  what you were originally
    proposing was a change of the wordage as follows:
    
    
    
                   In section 2.2.4 You suggested we change
    
    "An iSCSI initiator MUST send as unsolicited data either the negotiated
    amount or all the data if the total amount is less than the negotiated
    amount for unsolicited data."
    
                   to something like
    
    "An iSCSI initiator MAY choose to send no unsolicited data with a
    command,
    or if any unsolicited data is sent it MUST be either the negotiated
    amount
    or all the data if the total amount is less than the negotiated amount
    for
    unsolicited data."
    
    
    I, for one, have now come off the fence, and support your position, as
    long
    as it does not hold up the Draft.  Julian what do you think?
    
    
    
    
    
    .
    .
    .
    John L. Hufferd
    Senior Technical Staff Member (STSM)
    IBM/SSG San Jose Ca
    Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
    Home Office (408) 997-6136, Cell: (408) 499-9702
    Internet address: hufferd@us.ibm.com
    
    
    "Rod Harrison" <rod.harrison@windriver.com> on 04/10/2002 04:01:03 AM
    
    To:    John Hufferd/San Jose/IBM@IBMUS
    cc:    "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)"
           <matthew_burbridge@hp.com>, "'Julian Satran'"
           <Julian_Satran@il.ibm.com>, <ips@ece.cmu.edu>,
           <owner-ips@ece.cmu.edu>
    Subject:    RE: ISCSI: Unsolicited data in draft v12
    
    
    
    John,
    
     I'm confused by your assertion that "FirstBurstSize is always >= (the
    effective value of) MaxRecvPDULength". For unsolicited data it is true
    that FirstBurstSize limits the "effective" PDU size, but for solicited
    data the PDU size is min (MaxBurstSize, MaxRecvPDULength).
    
     It seems to be quite common for targets to advertise MaxBurstSize >
    MaxRecvPDULength > FirstBurstSize. For example we've seen values like
    MaxBurstSize=256k, MaxRecvPDULength=64k, and FirstBurstSize=16k at
    plugfests.
    
     Assuming unsolicited data are permitted, a SCSI transfer length of
    say 32k gives the initiator the option of either sending 16k
    unsolicited, either immediate or as DATA-IN, and then waiting for an
    R2T for the remainder of the payload, or just sending the command and
    shipping all the data in a single 32k PDU in response to a single R2T.
    The target has complete control over R2T mode of course, but when
    MaxRecvPDULength is greater than the SCSI transfer length it is
    reasonable to assume the target will issue a single R2T for the whole
    payload.
    
     - Rod
    
    -----Original Message-----
    From: John Hufferd [mailto:hufferd@us.ibm.com]
    Sent: Wednesday, April 10, 2002 10:42 AM
    To: Rod Harrison
    Cc: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2); 'Julian Satran';
    ips@ece.cmu.edu; owner-ips@ece.cmu.edu
    Subject: RE: ISCSI: Unsolicited data in draft v12
    
    
    
    Rod,
    FirstBurstSize is always >= (the effective value of) MaxRecvPDULength
    (as
    seen by the target).  So I still do not understand your point.
    MaxRecvPDULength is generally the buffer size on the Chip or HBA, and
    FirstBurstSize is the total amount of Main Memory reserved for the
    Unsolicited Data.
    
    I used the words, above, "effective value of" since MaxRecvPDULength
    is a
    connection value and FirstBurstSize is a Session Value.  So it might
    be
    that "this" connection could have a really big HBA buffer, one that
    was
    bigger then the Declared FirstBurstSize for the Session, however, the
    Maximum PDU Length is limited by the smaller of the FirstBurstSize or
    MaxBurstSize that was declared for the Session.  Therefore, I called
    it the
    effective value of MaxRecvPDULength.  However, all of this commentary
    does
    not change what I specified above.
    
    So is your point only focused on the times when the MaxRecvPDULength
    is
    equal to FirstBurstSize?  If so, then I do not see the issue. And if
    equal
    size is not your point, I am really missing it.
    
    .
    .
    .
    John L. Hufferd
    Senior Technical Staff Member (STSM)
    IBM/SSG San Jose Ca
    Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
    Home Office (408) 997-6136, Cell: (408) 499-9702
    Internet address: hufferd@us.ibm.com
    
    
    "Rod Harrison" <rod.harrison@windriver.com> on 04/09/2002 05:44:55 PM
    
    To:    John Hufferd/San Jose/IBM@IBMUS
    cc:    "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)"
           <matthew_burbridge@hp.com>, "'Julian Satran'"
           <Julian_Satran@il.ibm.com>, <ips@ece.cmu.edu>,
           <owner-ips@ece.cmu.edu>
    Subject:    RE: ISCSI: Unsolicited data in draft v12
    
    
    
    John,
    
     I agree, but this can only happen when FirstBurstSize is greater than
    MaxRecvPduSize. In that configuration I would expect the initiator to
    send
    unsolicited data.
    
     - Rod
    
    -----Original Message-----
    From: John Hufferd [mailto:hufferd@us.ibm.com]
    Sent: Wednesday, April 10, 2002 12:33 AM
    To: Rod Harrison
    Cc: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2); 'Julian Satran';
    ips@ece.cmu.edu; owner-ips@ece.cmu.edu
    Subject: RE: ISCSI: Unsolicited data in draft v12
    
    
    
    What I thought I said is that it should be just as efficient to send a
    normal (non immediate) unsolicited data-out PUD, following the
    immediate
    data, as it is to wait for an R2T and then send the data-out PDU, and
    probably a lot more efficient.  So I do not see why anyone would do
    other
    then that.
    
    .
    .
    .
    John L. Hufferd
    Senior Technical Staff Member (STSM)
    IBM/SSG San Jose Ca
    Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
    Home Office (408) 997-6136, Cell: (408) 499-9702
    Internet address: hufferd@us.ibm.com
    
    
    "Rod Harrison" <rod.harrison@windriver.com> on 04/09/2002 03:39:15 PM
    
    To:    John Hufferd/San Jose/IBM@IBMUS
    cc:    "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)"
           <matthew_burbridge@hp.com>, "'Julian Satran'"
           <Julian_Satran@il.ibm.com>, <ips@ece.cmu.edu>,
           <owner-ips@ece.cmu.edu>
    Subject:    RE: ISCSI: Unsolicited data in draft v12
    
    
    
    
     My reading of the new rule was that it covered both immediate and
    non-immediate
    unsolicited data. I think though the logic in my example would still
    apply,
    there is probably an efficiency gain in sending all the data in single
    solicited
    DATA-OUT versus 8k in an unsolicited DATA-OUT and 4k in a solicited
    DATA-OUT.
    Just thinking in terms of DMA operations on the initiator HBA I
    suspect
    there is
    an advantage to a single DMA for 12k over one for 8k followed by one
    for
    4k. I
    am assuming here that the initiator can not speculatively DMA ahead
    because
    of
    buffer space constraints.
    
     John, I'm not quite sure what you meant in your second sentence. Are
    you
    saying
    even with the new rule the initiator can choose to not send immediate
    data
    even
    if it has been negotiated to be available?
    
     - Rod
    
    -----Original Message-----
    From: John Hufferd [mailto:hufferd@us.ibm.com]
    Sent: Tuesday, April 09, 2002 11:20 PM
    To: Rod Harrison
    Cc: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2); 'Julian Satran';
    ips@ece.cmu.edu; owner-ips@ece.cmu.edu
    Subject: RE: ISCSI: Unsolicited data in draft v12
    
    
    
    I understand what you said, however, I thought the issue was for non
    immediate unsolicited data.  The case you made is normal, I think, for
    the
    choice for immediate vrs non immediate (either R2T or other
    unsolicited
    data).
    If the Initiator has agreed to use non immediate unsolicited data then
    it
    is not clear, using your example, why one would not send the data via
    a
    normal unsolicited Data-Out PDU, when ready.
    
    .
    .
    .
    John L. Hufferd
    Senior Technical Staff Member (STSM)
    IBM/SSG San Jose Ca
    Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
    Home Office (408) 997-6136, Cell: (408) 499-9702
    Internet address: hufferd@us.ibm.com
    
    
    "Rod Harrison" <rod.harrison@windriver.com> on 04/09/2002 12:28:27 PM
    
    To:    John Hufferd/San Jose/IBM@IBMUS, "BURBRIDGE,MATTHEW
           (HP-UnitedKingdom,ex2)" <matthew_burbridge@hp.com>
    cc:    "'Julian Satran'" <Julian_Satran@il.ibm.com>,
    <ips@ece.cmu.edu>,
           <owner-ips@ece.cmu.edu>
    Subject:    RE: ISCSI: Unsolicited data in draft v12
    
    
    
    
     As John has guessed my thinking, at least in part, was of dealing
    with
    congestion in the HBA. I can imagine there might be times when the
    initiator receives a command and cannot provide buffer space to
    accommodate
    the unsolicited data, in which case it might be able to ship the
    command
    and deal with the data when the target sends R2Ts.
    
     I also think there might be efficiency gains to be had by allowing
    the
    initiator to ship commands without immediate data in some instances.
    Consider the case where MaxRecvPduSize for the target is 64k and
    FirstBurstSize is 8k, we've seen this sort of thing quite a bit at
    plugfests. If the initiator receives a SCSI command with a transfer
    length
    of 12k it is probably more efficient to ship the command immediately
    and
    then send the whole payload in one DATA-OUT in response to a single
    R2T
    from the target, than to ship 8k of immediate data and then 4k in
    response
    to an R2T. In fact for any SCSI transfer length greater than
    FirstBurstSize
    and less than MaxRecvPduSize this is probably true.
    
     I response to Mathew's concern about pre-allocated buffer space being
    wasted if unsolicited data isn't sent, I believe this might be more of
    a
    theoretical concern than a practical one. Any buffer space that is set
    aside for unsolicited data that can't be used for anything else will
    be
    wasted on every command where the SCSI transfer length is greater than
    FurstBurstSize when the task moves into "R2T mode" anyway. For large
    transfers that could be a significant proportion of the time. Consider
    a
    modest 12 MB transfer with a generous FurstBurstSize of 1MB, that
    leaves 11
    MB of payload to be transferred under the auspices of R2T, during
    which
    time the unsolicited buffers are unavailable. Multiply that by the
    command
    window size, and then by the number of initiators the target might
    service
    and you quickly end up with an impossible situation.
    
     - Rod
    
    -----Original Message-----
    From: John Hufferd [mailto:hufferd@us.ibm.com]
    Sent: Tuesday, April 09, 2002 12:41 AM
    To: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)
    Cc: 'Julian Satran'; BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2);
    ips@ece.cmu.edu; owner-ips@ece.cmu.edu; Rod Harrison
    Subject: RE: ISCSI: Unsolicited data in draft v12
    
    
    
    Matthew,
    I would have thought that if there is some special buffer space set
    aside
    for the session, whether physical set aside or as a high/low water
    mark, it
    would still be available for other tasks in the session, even if some
    tasks
    do not use it, so I fail to see the true impact.
    
    Perhaps you have seen something or fear something that I do not
    understand
    about why a Initiator would negotiate the unsolicited buffer space
    (FirstBustSize) and then not use it, except for when it had some kind
    of
    congestion, or the like.
    
    If you state why you think this would happen, perhaps those persons
    (Rod)
    that want this "MUST" changed to "MAY", should state why they think it
    is
    important to them.
    
    I actually do not see the point of either side.
    
    .
    .
    .
    John L. Hufferd
    Senior Technical Staff Member (STSM)
    IBM/SSG San Jose Ca
    Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
    Home Office (408) 997-6136, Cell: (408) 499-9702
    Internet address: hufferd@us.ibm.com
    
    
    "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" <matthew_burbridge@hp.com>
    on
    04/08/2002 03:44:47 PM
    
    To:    "'Julian Satran'" <Julian_Satran@il.ibm.com>,
    "BURBRIDGE,MATTHEW
           (HP-UnitedKingdom,ex2)" <matthew_burbridge@hp.com>
    cc:    ips@ece.cmu.edu, John Hufferd/San Jose/IBM@IBMUS,
    "BURBRIDGE,MATTHEW
           (HP-UnitedKingdom,ex2)" <matthew_burbridge@hp.com>,
           owner-ips@ece.cmu.edu, Rod Harrison
    <rod.harrison@windriver.com>
    Subject:    RE: ISCSI: Unsolicited data in draft v12
    
    
    
    It would not necessarily need separate buffers but it does need to
    keep
    some
    buffers pre-allocated for unsolicited data so when the data arrives
    unsolicited there is a buffer available in which to place the data.
    
    Matthew
    
    
    -----Original Message-----
    From: Julian Satran [mailto:Julian_Satran@il.ibm.com]
    Sent: Monday, April 08, 2002 12:00 PM
    To: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)
    Cc: ips@ece.cmu.edu; 'John Hufferd'; BURBRIDGE,MATTHEW
    (HP-UnitedKingdom,ex2); owner-ips@ece.cmu.edu; Rod Harrison
    Subject: RE: ISCSI: Unsolicited data in draft v12
    
    
    
    I am with John here (the third guy that is right) - why would an
    implementer
    have separate buffers for solicited and unsolicited data?
    
    Julo
    
    
    "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" <matthew_burbridge@hp.com>
    08-04-02 21:43
    Please respond to "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)"
    
            To:        John Hufferd/San Jose/IBM@IBMUS, "BURBRIDGE,MATTHEW
    (HP-UnitedKingdom,ex2)" <matthew_burbridge@hp.com>
            cc:        Julian Satran/Haifa/IBM@IBMIL, Rod Harrison
    <rod.harrison@windriver.com>, ips@ece.cmu.edu, owner-ips@ece.cmu.edu
            Subject:        RE: ISCSI: Unsolicited data in draft v12
    
    
    
    
    John,
    
    It's not so much an implementation problem but one resource management
    problem in that if unsolicited data has been negotiated then target
    MUST
    pre-allocate buffers with which to store the unsolicited when it
    arrives.
    The target implementors will decided whether they want to use
    unsolicted
    data and take the buffer resource hit in doing so.  However, if they
    do
    wish
    to take this hit but the initators decide not to use unsolicited data
    (even
    though they have negotiated to use it) then there is potientially a
    lot of
    valuable buffer resources tied in up for unsolicited data but which is
    not
    being used.
    
    Matthew
    
    -----Original Message-----
    From: John Hufferd [mailto:hufferd@us.ibm.com]
    Sent: Monday, April 08, 2002 11:13 AM
    To: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)
    Cc: 'Julian Satran'; Rod Harrison; ips@ece.cmu.edu;
    owner-ips@ece.cmu.edu
    Subject: RE: ISCSI: Unsolicited data in draft v12
    
    
    
    Please explain, why an initiator deciding to not send unsolicited data
    for
    a specific command causes an implementation problem.  That was not
    clear
    from your statements.  You still need the R2T capability, so what is
    lost?
    
    .
    .
    .
    John L. Hufferd
    Senior Technical Staff Member (STSM)
    IBM/SSG San Jose Ca
    Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
    Home Office (408) 997-6136, Cell: (408) 499-9702
    Internet address: hufferd@us.ibm.com
    
    
    "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" <matthew_burbridge@hp.com>
    @ece.cmu.edu on 04/08/2002 10:25:55 AM
    
    Sent by:    owner-ips@ece.cmu.edu
    
    
    To:    "'Julian Satran'" <Julian_Satran@il.ibm.com>, Rod Harrison
          <rod.harrison@windriver.com>
    cc:    ips@ece.cmu.edu, owner-ips@ece.cmu.edu
    Subject:    RE: ISCSI: Unsolicited data in draft v12
    
    
    
    I must express my concern on this issue.  From a target point of view
    once
    it has negoiated the use of unsolicited data it has to allocate buffer
    space
    for that unsolicited data.  Now depending on the various parameters
    this
    may
    be a sizeable chunk of valuable resources which it is making
    available.
    Now
    if the decision to use unsolicited data is being moved from a per
    session
    to
    per task basis (which is what this change effectively does) then it
    puts an
    awful lot of resource overhead on the target which may never be used.
    
    For the reasons above I propose that we do not relax the v12
    restriction
    and
    keep it as:
    
    "An iSCSI initiator MUST send as unsolicited data either the
    negotiated
    amount or all the data if the total amount is less than the negotiated
    amount for unsolicited data."
    
    Matthew Burbridge
    Principal Engineer
    NSAS-Bristol
    Hewlett Packard
    
    -----Original Message-----
    From: Julian Satran [mailto:Julian_Satran@il.ibm.com]
    Sent: Monday, April 08, 2002 9:36 AM
    To: Rod Harrison
    Cc: ips@ece.cmu.edu; owner-ips@ece.cmu.edu
    Subject: Re: ISCSI: Unsolicited data in draft v12
    
    
    
    OK - Julo
    
    
    "Rod Harrison" <rod.harrison@windriver.com>
    Sent by: owner-ips@ece.cmu.edu
    08-04-02 14:52
    Please respond to "Rod Harrison"
    
           To:        <ips@ece.cmu.edu>
           cc:
           Subject:        ISCSI: Unsolicited data in draft v12
    
    
    
    
    
                   I propose we slightly relax the new restriction in
    draft
    v12
    that the
    initiator MUST send the maximum permissible amount of unsolicited
    data. I
    suggest we change the rule to allow the initiator to either send no
    unsolicited data, or the maximum permissible.
    
                   There is no difficulty for the target here since the
    lack
    of
    unsolicited
    data will be clearly indicated by a command PDU with F bit set and
    dataSegLen=0. The target will have all the information it needs to
    immediately issue R2Ts as appropriate.
    
                   I believe the initiator should be able to make a policy
    decision on which
    individual commands should be sent with unsolicited data and which
    should
    not.
    
                   In draft 11.91 section 2.2.4 I suggest we change
    
    "An iSCSI initiator MUST send as unsolicited data either the
    negotiated
    amount or all the data if the total amount is less than the negotiated
    amount for unsolicited data."
    
                   to something like
    
    "An iSCSI initiator MAY choose to send no unsolicited data with a
    command,
    or if any unsolicited data is sent it MUST be either the negotiated
    amount
    or all the data if the total amount is less than the negotiated amount
    for
    unsolicited data."
    
                   - Rod
    
    
    
    
    
    
    
    
    
    
    
    
    
    


Home

Last updated: Thu Apr 11 16:18:24 2002
9609 messages in chronological order