SORT BY:

LIST ORDER
THREAD
AUTHOR
SUBJECT


SEARCH

IPS HOME


    [Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

    Re: iSCSI: Markers




    One comment in text - julo


    Paul Koning <pkoning@equallogic.com>
    Sent by: owner-ips@ece.cmu.edu

    08-01-02 18:59

           
            To:        John Hufferd/San Jose/IBM@IBMUS
            cc:        Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu
            Subject:        Re: iSCSI: Markers

           


    >>>>> "John" == John Hufferd <hufferd@us.ibm.com> writes:

    John> OK, Julian, I buy that argument.  I think that you have made
    John> the point that the value of Markers is Greater then I was
    John> giving credit for, in iSCSI/TOE integrated HBAs that are
    John> operating in CRC mode.  That is great, since I am afraid we
    John> will not see the Framing stuff for some time.

    John> So the questions are which form of markers is the best.

    John> My vote comes down on the side of Fixed Interval Markers
    John> (FIM). For the following reasons:

    John> On the SW sending side, FIM works with low overhead whether CRC
    John> is being used or not.  COWS, is a dramatic overhead unless the
    John> SW is performing CRC computation anyway.

    John> The COWS has a built in conflict with the Pipeline HW HBA, that
    John> has two different needs regarding the direction of the Pointers
    John> when talking to another partner HW HBA that uses a Pipeline
    John> approach.

    I think you are making general assumptions about what is cheap in
    software that may be valid for SOME software implementations but are
    certainly not valid for others.

    In particular, FIM is NOT necessarily low overhead.  Scatter/gather is
    a reasonably cheap mechanis IF it is available for this job.  (Note
    that it is not all that cheap: adding a bunch of extra descriptor
    fetches to the I/O device's job is extra overhead, the scale of which
    depends on the hardware.

    The problem is that scatter/gather may not be available at all.  If it
    is available in principle, it may have restrictions that clash with
    what FIM needs.  (For example, a particular implementation may support
    breaking buffers up in many pieces, but all breaks must be on cache
    line boundaries.  For a cache line size of 32, clearly you cannot use
    scatter/gather in that implementation to do FIM.)

    So FIM may force a full buffer copy, which is a major hit.

    As for COWS, it's clearly nasty.  Even if you're doing CRC, it adds a
    significant increment to the processing cost.

    In conclusion, I am opposed to mandating markers.  Adding such a large
    increment in processing overhead doesn't make sense to me, given that
    I don't see how it can be expected to save all that much memory in
    implementations that use it.  (As far as I can see, the gain doesn't
    come anywhere near a factor of two.  Is there any analysis available
    that discusses this in any detail -- covering not just the specific
    buffering needs of the reassembly side but also the other memory
    requirements in a node?  It seems to me a node needs at least as much
    memory in the other direction -- quite possibly more if it's the
    storage end, given that a lot of the traffic is reads.)


    +++ the gain is not a factor of two - it is a not a constant - it is rather
    proportional to the bandwidth delay product.
    The rational is simple - once a header is lost without any framing mechanism
    the receiver has to accumulate data until the missing piece is recovered.
    This and other lists had long discussion threads on the subject - and if you care about it you may want to go over the archives.
    As David has said it is generaly agreed that this is not an issue at 1Gbs yet - but is a also agreed that this is a major issue at 10 Gb/s.
    That is why some of us think that a stronger statement about some form of framing is required.

    +++

                    paul




Home

Last updated: Thu Jan 10 00:17:51 2002
8332 messages in chronological order