SORT BY:

LIST ORDER
THREAD
AUTHOR
SUBJECT


SEARCH

IPS HOME


    [Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

    iSCSI: Out of order commands, was current UNH Plugfest



    
    	[ Subject changed ]
    
    Julian,
    
    	The ordering difference is introduced between the host side driver
    and the iSCSI HBA. The host side driver must present SCSI commands to
    the HBA in the order they are received from the OS to prevent read
    after write dependency failures. The HBA might reorder the commands
    depending on when DMA completes. The reordering can't be done ahead of
    time in the host driver since it doesn't know how long each DMA might
    take. As long as the HBA assigns CmdSN in the order it receives
    commands the desired host ordering is preserved.
    
    	- Rod
    
    -----Original Message-----
    From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
    Julian Satran
    Sent: Monday, November 05, 2001 12:35 AM
    To: ips@ece.cmu.edu
    Subject: RE: iSCSI: current UNH Plugfest
    
    
    Rod,
    
    I all examples give the point I find hard to understand is why is the
    ordering on the wire different from the presentation order to the
    initiator.  You can get as many overlaps as you want by presenting the
    commands to the initiator in the desired order.
    What we are considering here is the case in which you want to ship in
    an
    order different than the one you present the commands.
    
    Julo
    
    
    
    
    "Rod Harrison" <rod.harrison@windriver.com>
    Sent by: owner-ips@ece.cmu.edu
    04-11-01 04:42
    Please respond to "Rod Harrison"
    
    
            To:     "Barry Reinhold" <bbrtrebia@mediaone.net>, "Dave
    Sheehy"
    <dbs@acropora.rose.agilent.com>, "IETF IP SAN Reflector"
    <ips@ece.cmu.edu>
            cc:
            Subject:        RE: iSCSI: current UNH Plugfest
    
    
    
    Barry,
    
                     In general I agree but I don't think this is as much
    of a
    corner case
    as it at first appears. Targets will have code very similar to that
    needed to handle out of order commands to deal with digest errors.
    Targets also need to queue commands whilst waiting for both solicited
    and unsolicited data to arrive. Queuing out of order commands seems
    little extra work.
    
                     From an initiators point of view there are
    efficiency,
    and probably
    performance gains to be had from sending commands out of order. Bob
    Russell gave the example of a read being sent whilst write data DMA is
    happening, and a similar situation can arise with DMA for writes
    overtaking that of earlier writes if the initiator has multiple DMA
    engines. In this case the initiator might be forced to let the wire go
    idle if it can't send the data from completed DMAs as soon as
    possible.
    
                     We already have a command queue at the target to
    enforce
    correct
    serialisation of commands, doing the same thing at the initiator is
    redundant.
    
                     Finally, I don't believe we should be writing a
    standard
    to work
    around poor coding and test coverage, especially at the cost of
    potential efficiency gains.
    
                     I agree with Dave and Santosh that commands being
    sent
    out of order
    on a single session should be allowed by the standard.
    
                     - Rod
    
    -----Original Message-----
    From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
    Barry Reinhold
    Sent: Friday, November 02, 2001 5:24 PM
    To: Dave Sheehy; IETF IP SAN Reflector
    Subject: RE: iSCSI: current UNH Plugfest
    
    
    Using features such as out of order command delivery on a connection
    tend to
    be the sort of things that lead to interoperability problems. It is
    unexpected and probably going to hit poorly tested code paths even if
    the
    standard is written to allow it.
    
    >-----Original Message-----
    >From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf
    Of
    >Dave Sheehy
    >Sent: Friday, November 02, 2001 4:19 PM
    >To: IETF IP SAN Reflector
    >Subject: Re: iSCSI: current UNH Plugfest
    >
    >
    >
    >> 3. Can commands be sent out of order on the same connection?
    >>
    >>    The behavior of targets is clearly specified in Section 2.2.2.3
    on
    >>    page 25 of draft 8, which says:
    >>      "Except for the commands marked for immediate delivery the
    iSCSI
    >>      target layer MUST eliver the commands for execution in the
    order
    >>      specified by CmdSN."
    >>
    >>    Section 2.2.2.3 on page 26 of draft 8 also says:
    >>      "- CmdSN - the current command Sequence Number advanced by 1
    on
    >>      each command shipped except for commands marked for immediate
    >>      delivery."
    >>    but the meaning of the term "shipped" is vague, and does not
    >> necessarily
    >>    require that the PDUs arrive on the other end of a TCP
    connection
    >>    in the same order that the CmdSN values were assigned to these
    PDUs.
    >>
    >>    Some initiators have been designed to send commands out of CmdSN
    >>    order on one connection.  Consider the situation where there is
    only
    >>    one connection and a high-level dispatcher creates a PDU for a
    SCSI
    >>    command that involves writing immediate data to the target.
    This PDU
    >>    is enqueued to a lower-level layer which has to setup, start,
    and
    >>    wait-for a DMA operation to move the immediate data into an
    onboard
    >>    buffer before the PDU can be put onto the wire.  While this is
    >>    happening, the dispatcher creates another unrelated PDU for a
    SCSI
    >>    read command (for example), and when this PDU is passed to the
    >>    lower-level layer it can be sent immediately, ahead of the
    previous
    >>    write PDU and therefore out of order on this connection.
    >>
    >>    The standard clearly allows this to happen if the two PDUs were
    sent
    >>    on different connections, and seems to imply that this can also
    happen
    >>    when the two PDUs are sent on the same connection.
    >>
    >>    The suggestion is to put in the standard an explicit statement
    that
    >>    this is allowed or not allowed, as appropriate.
    >>
    >>    If this is allowed, such a statement would avoid the erroneous
    >>    assumption being made by some target implementers that within a
    single
    >>    connection, commands will arrive in order.
    >>
    >>    If this is not allowed, such a statement would avoid the
    erroneous
    >>    assumption being made by some initiator implementers that within
    a
    >>    single connection, commands can be put on the wire out of order.
    >>
    >> +++
    >>
    >> will add an explicit statement saying that this behaviour is
    forbidden.
    >> 2.2.2.1 will contain:
    >>
    >> On any given connection, the iSCSI initiator MUST send the
    >commands in the
    >> order specified by CmdSN.
    >>
    >> +++
    >
    >Why do you feel this behavior should be forbidden? Targets already
    have to
    >order commands across the session. I don't see why it's a problem to
    extend
    >that to the connection as well. I, for one, believe we should take
    >a liberal
    >stance on this.
    >
    >Dave Sheehy
    >
    
    
    
    
    


Home

Last updated: Mon Nov 05 15:17:33 2001
7560 messages in chronological order