SORT BY:

LIST ORDER
THREAD
AUTHOR
SUBJECT


SEARCH

IPS HOME


    [Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

    Re: iSCSI: Markers



    >>>>> "John" == John Hufferd <hufferd@us.ibm.com> writes:
    
     John> OK, Julian, I buy that argument.  I think that you have made
     John> the point that the value of Markers is Greater then I was
     John> giving credit for, in iSCSI/TOE integrated HBAs that are
     John> operating in CRC mode.  That is great, since I am afraid we
     John> will not see the Framing stuff for some time.
    
     John> So the questions are which form of markers is the best.
    
     John> My vote comes down on the side of Fixed Interval Markers
     John> (FIM). For the following reasons:
    
     John> On the SW sending side, FIM works with low overhead whether CRC
     John> is being used or not.  COWS, is a dramatic overhead unless the
     John> SW is performing CRC computation anyway.
    
     John> The COWS has a built in conflict with the Pipeline HW HBA, that
     John> has two different needs regarding the direction of the Pointers
     John> when talking to another partner HW HBA that uses a Pipeline
     John> approach.
    
    I think you are making general assumptions about what is cheap in
    software that may be valid for SOME software implementations but are
    certainly not valid for others.
    
    In particular, FIM is NOT necessarily low overhead.  Scatter/gather is
    a reasonably cheap mechanis IF it is available for this job.  (Note
    that it is not all that cheap: adding a bunch of extra descriptor
    fetches to the I/O device's job is extra overhead, the scale of which
    depends on the hardware.
    
    The problem is that scatter/gather may not be available at all.  If it
    is available in principle, it may have restrictions that clash with
    what FIM needs.  (For example, a particular implementation may support
    breaking buffers up in many pieces, but all breaks must be on cache
    line boundaries.  For a cache line size of 32, clearly you cannot use
    scatter/gather in that implementation to do FIM.)
    
    So FIM may force a full buffer copy, which is a major hit.
    
    As for COWS, it's clearly nasty.  Even if you're doing CRC, it adds a
    significant increment to the processing cost.
    
    In conclusion, I am opposed to mandating markers.  Adding such a large
    increment in processing overhead doesn't make sense to me, given that
    I don't see how it can be expected to save all that much memory in
    implementations that use it.  (As far as I can see, the gain doesn't
    come anywhere near a factor of two.  Is there any analysis available
    that discusses this in any detail -- covering not just the specific
    buffering needs of the reassembly side but also the other memory
    requirements in a node?  It seems to me a node needs at least as much
    memory in the other direction -- quite possibly more if it's the
    storage end, given that a lot of the traffic is reads.)
    
    	paul
    
    


Home

Last updated: Wed Jan 09 09:18:03 2002
8324 messages in chronological order