SORT BY:

LIST ORDER
THREAD
AUTHOR
SUBJECT


SEARCH

IPS HOME


    [Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

    RE: iSCSI: Use of the A bit



    I cannot negotiate the maxBurstSize to a "nice" size because regardless of
    what size is chosen, the Ios that I am processing will not line up nicely on
    maxBurstSize boundaries.
    
    Consider:
    
    Assume the MaxBurstSize = 128K, This is the only IO on the connection.
    
    1. The initiator sends a 64K read request to the target.
    2. The target sends 64K to the initiator. Now it wants to know that all of
    the data packets have been received so that resources for the IO can be
    released and the IO complete sent to the OS. I cannot set the A-Bit on the
    final data PDU because that would violate the clause that says:
    
    "it MAY set the A-bit to 1 once at most every
    MaxBurstSize bytes, and MUST NOT do so any more frequently than that."
    
    But since I have the entire IO completed, I could send a Nop-Ping. That
    would force the initiator to send the new StatSN which would imply all data
    packets have been received. 
    
    But there is also a more complicated case. Our chip allows the higher level
    software to break IOs into its own data phases. If more than one phase is
    used, then a response is not the last PDU sent and there is no work around
    to get the data Ack'd.
    
    Furthermore, negotiating the burstsize low so that I can set the a_bit when
    I want has the undesired side effect of allowing only that much data per R2T
    request. A definite performance squelcher.
    
    Kevin Lemay
    
    -----Original Message-----
    From: John Hufferd [mailto:hufferd@us.ibm.com]
    Sent: Thursday, March 14, 2002 11:36 AM
    To: LEMAY,KEVIN (A-Roseville,ex1)
    Cc: 'Eddy Quicksall'; BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2); 'Rod
    Harrison'; ips@ece. cmu. edu (E-mail)
    Subject: RE: iSCSI: Use of the A bit
    
    
    
    We got to this position, since so many folks did not want to support the
    positive ack.  And this approach was put in as a compromise.  I would
    suggest, that the thing we are chasing is a small corner case that may not
    be smart enough to operate at anything other then ErrorRecoverylevel=0.  In
    any event I do not see a problem with limiting the response to MaxBurstSize
    intervals.  If the interval is too big, why did the unit negotiate it to
    that size?
    
    .
    .
    .
    John L. Hufferd
    Senior Technical Staff Member (STSM)
    IBM/SSG San Jose Ca
    Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
    Home Office (408) 997-6136, Cell: (408) 499-9702
    Internet address: hufferd@us.ibm.com
    
    
    "LEMAY,KEVIN (A-Roseville,ex1)" <kevin_lemay@agilent.com>@ece.cmu.edu on
    03/14/2002 10:43:31 AM
    
    Sent by:    owner-ips@ece.cmu.edu
    
    
    To:    "'Eddy Quicksall'" <Eddy_Quicksall@ivivity.com>, "BURBRIDGE,MATTHEW
           (HP-UnitedKingdom,ex2)" <matthew_burbridge@hp.com>, "'Rod Harrison'"
           <rod.harrison@windriver.com>, John Hufferd/San Jose/IBM@IBMUS
    cc:    "ips@ece. cmu. edu (E-mail)" <ips@ece.cmu.edu>
    Subject:    RE: iSCSI: Use of the A bit
    
    
    
    I also do not like the comment about having to wait until maxBurstsize byte
    to set the A-bit.
    
    I wanted to use it as Eddie suggested. To force the initiator to
    acknowledge
    the IO so that I can release resources on the target.
    
    This will be especially useful for hardware accelerated IOs where the chip
    cannot declare the IO complete until it has been Ack'd.
    
    Could we just change the wording to:
    ----
    For sessions with ErrorRecoveryLevel 1 or higher the target sets this bit
    to
    1 to indicate that it requests from the initiator a positive
    acknowledgement
    for the data received. The target should use the A bit moderately; the
    A-bit
    MAY only be set to 1 at the end of a data sequence.
    
    On receiving a Data-In PDU with the A bit set to 1 the initiator MUST
    issued
    a SNACK of type DataACK. If the initiator has detected holes in the input
    sequence, it MUST postpone issuing the SNACK of type ACKN until the holes
    are filled.
    ---
    
    This way I can always set the A bit at the end of an IO.
    
    Kevin Lemay
    
    -----Original Message-----
    From: Eddy Quicksall [mailto:Eddy_Quicksall@ivivity.com]
    Sent: Thursday, March 14, 2002 5:51 AM
    To: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2); 'Rod Harrison'; John
    Hufferd
    Cc: ips@ece. cmu. edu (E-mail)
    Subject: RE: iSCSI: Use of the A bit
    
    
    I see a lot of responses talking about ErrorRecoveryLevel = 0. That was not
    my original point. ErrorRecoveryLevel = 0 already has this solved. I am
    talking about ErrorRecoveryLevel > 0 (I made an error in my first EMAIL and
    the point may have been lost).
    
    If the initiator has the option to ignore the A bit, the target will get
    stuck. What is wrong with requiring the initiator to respond? Can we just
    get that answered first?
    
    I suspect that there is a concern that this may cause too much traffic. My
    answer would be "don't buy a cheep target ... their lack of memory may
    cause
    degraded performance" (not intended for the spec, of course :-) ).
    
    I also do not see the rational in not allowing the A bit more often than
    MaxBurstSize. The ULP layers may not know anything about the transport
    layers. So their resource issues are not known by the transport(s). The
    MaxBurstSize is an iSCSI issue and may not apply to future protocols.
    
    So, why not just say something like:
    
    1) The initiator MUST send a DataACK SNACK if it sees the A bit set (and if
    ErrorRecoveryLevel > 0)
    2) The A bit may be set only when the F bit is set.
    
    And leave it go at that?
    
    Eddy
    
    -----Original Message-----
    From: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)
    [mailto:matthew_burbridge@hp.com]
    Sent: Thursday, March 14, 2002 7:22 AM
    To: 'Rod Harrison'; John Hufferd
    Cc: Eddy Quicksall; ips@ece. cmu. edu (E-mail)
    Subject: RE: iSCSI: Use of the A bit
    
    
    I think that what Santosh and others have stated is the correct think to do
    for initiators.  If the revovery level is 0 and the A bit is set then the
    initiator is quite with in its rights to ignore the A bit (If the target is
    at fault then penalise the target and not the initiator).
    
    IMHO changing the spec to make it more specific is not the right thing and
    it could lead to inter operability problems.  However, I do think something
    needs to be added so I propose adding the statement:
    
    "If ErrorRecoveryLevel=0 then the initiator MAY ignore the A bit (i.e.
    assume A=0)".
    
    One could argue for the MAY to be a MUST but if there is a kind initiator
    out there then let it do its stuff - it can only help the rogue target.
    
    Matthew
    
    -----Original Message-----
    From: Rod Harrison [mailto:rod.harrison@windriver.com]
    Sent: Thursday, March 14, 2002 9:27 AM
    To: John Hufferd
    Cc: Eddy Quicksall; BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2); ips@ece.
    cmu. edu (E-mail)
    Subject: RE: iSCSI: Use of the A bit
    
    
    John,
    
     Actually 9.7.2 says almost exactly that ...
    
     "For sessions with ErrorRecoveryLevel 1 or higher, the target sets
    this bit to 1 to indicate that it requests a positive acknowledgement
    from the initiator for the data received."
    
     We've interpreted that to mean our initiator should ignore the A bit
    if ErrorRecoveryLevel is 0.
    
     Having said that it is very easy to support just the positive form
    of
    SNACK, so I would have no objection to removing the A bits current
    link to ErrorRecoveryLevel.
    
     - Rod
    
    -----Original Message-----
    From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
    John Hufferd
    Sent: Thursday, March 14, 2002 3:12 AM
    To: Rod Harrison
    Cc: Eddy Quicksall; BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2); ips@ece.
    cmu. edu (E-mail)
    Subject: RE: iSCSI: Use of the A bit
    
    
    
    Actually it does not actually say that.  It says "This is limited to
    sessions that support error recovery and is implemented through the A
    bit
    ..."
    
    I know what I am about to say, is nit picking, but ....
    ErrorRecoveryLevel=0 is an error recovery technique.
    
    Having said that, we probably should just change that statement to
    "This is
    implemented through the A bit ..."
    
    
    .
    .
    .
    John L. Hufferd
    Senior Technical Staff Member (STSM)
    IBM/SSG San Jose Ca
    Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
    Home Office (408) 997-6136, Cell: (408) 499-9702
    Internet address: hufferd@us.ibm.com
    
    
    "Rod Harrison" <rod.harrison@windriver.com>@ece.cmu.edu on 03/13/2002
    01:05:51 PM
    
    Sent by:    owner-ips@ece.cmu.edu
    
    
    To:    "Eddy Quicksall" <Eddy_Quicksall@ivivity.com>,
    "BURBRIDGE,MATTHEW
           (HP-UnitedKingdom,ex2)" <matthew_burbridge@hp.com>, "ips@ece.
    cmu.
           edu (E-mail)" <ips@ece.cmu.edu>
    cc:
    Subject:    RE: iSCSI: Use of the A bit
    
    
    
    Eddy,
    
     You were right first time, in draft 11 section 2.5.1.5 paragraph 5
    and 9.7.2 state that the A-bit is only supported if ErrorRecoveryLevel
    is 1 or higher.
    
     - Rod
    
    -----Original Message-----
    From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
    Eddy Quicksall
    Sent: Wednesday, March 13, 2002 8:03 PM
    To: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2); ips@ece. cmu. edu
    (E-mail)
    Subject: RE: iSCSI: Use of the A bit
    
    
    Then my only concern is that the initiator may ignore the A bit if it
    deems that the bit is being set aggressively.
    
    If it ignored it, then the target would be stalled waiting for he ACK.
    
    Eddy
    -----Original Message-----
    From: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)
    [mailto:matthew_burbridge@hp.com]
    Sent: Wednesday, March 13, 2002 1:39 PM
    To: 'Eddy Quicksall'; ips@ece. cmu. edu (E-mail)
    Subject: RE: iSCSI: Use of the A bit
    Importance: High
    
    
    Eddy,
    
    The target is quite within its rights to use the A bit when at
    recovery level 0.  If the session is re-established due to recovery
    7.11.4 then the relevant command is aborted anyway and so there is no
    reason to keep hold of the data any way: With recovery level 0 there
    is no recovery mechanism that requires the target to keep the data.
    Therefore the A bit is redundant when the recovery level is 0.
    
    The spec says that the initiator MUST issue a SNACK if the A bit is
    set.  However, the MaxBurstSize restriction is there to prevent the
    initiator from having to send a SNACK on every PDU in the case where a
    target inadvertently sets the A bit in (for example) every data in
    PDU. The target may set the A bit more often than the MaxBurstSize but
    it should not expect a SNACK more often than this.
    
    Matthew Burbridge
    -----Original Message-----
    From: Eddy Quicksall [mailto:Eddy_Quicksall@ivivity.com]
    Sent: Wednesday, March 13, 2002 3:12 PM
    To: ips@ece. cmu. edu (E-mail)
    Subject: iSCSI: Use of the A bit
    
    
    Here is a case that I want to go over and if there is not already a
    solution, I think a refinement to the A bit could solve it.
    
    The problem is that a target (perhaps an iSCSI disk drive) does not
    have enough memory to transfer the full READ request so it must read
    from the medium as much as it can, transmit that, when that
    transmission is known to be good, read the next bunch, transmit that
    and so on.
    
    The problem we have is that the target must keep the buffer around
    until the transfer has been "ack'd" via ExpStatSN. But that status
    can't be sent because all of the requested data has not been sent. So
    the target would have to refuse to do the command.
    
    I was going to use the A bit for this thinking it would force the
    initiator to give an "ack" but our current wording does not make this
    a sure fire thing:
    
    1) The initiator may not want to run at ErrorRecoveryLevel 1.
    2) The initiator may ignore the A bit if it deems that the bit is
    being set aggressively.
    3) The target may set the A bit no more frequently than MaxBurstSize.
    
    Comments?
    
    mailto:Eddy_Quicksall@iVivity.com
    
    
    
    
    


Home

Last updated: Thu Mar 14 15:18:08 2002
9123 messages in chronological order