SORT BY:

LIST ORDER
THREAD
AUTHOR
SUBJECT


SEARCH

IPS HOME


    [Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

    RE: iSCSI: No Framing



    Jim,
    
    > -----Original Message-----
    > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
    > Jim Pinkerton
    > Sent: Tuesday, February 05, 2002 6:53 AM
    > To: John Hufferd; somesh_gupta@silverbacksystems.com
    > Cc: Mukund, Shridhar; ips@ece.cmu.edu
    > Subject: RE: iSCSI: No Framing
    > 
    > 
    > 
    > In reviewing the mail alias, and after having many conversations with
    > various folks, I would like to retract my prior opinion on ripping out
    > markers. It appears to me as though _every_ NIC vendor is in support of
    > markers. 
    
      Every NIC vendor may implement markers (or as is obvious from
      interoperability results, may have the capability to implement
      markers). On whether every (person who works at a) NIC/Chip vendor
      supports it from a technical perspective (i.e. whether it solves
      anything, or is a useful solution, or whether the analysis people
      are using is correct) supports it, I think the count seems to be
      evenly split.
    > 
    > To me the evidence in support of keeping in "Sync and Steering with
    > Fixed Interval Markers" is pretty compelling (now that I've been
    > re-educated). Without going into proprietary issues - how often do NIC
    > vendors agree unanimously on something? Also, a software implementation
    > of markers is tractable.
    > 
    > I have not heard a single NIC vendor in support of COWS, and I
    > personally don't support COWS either. Thus I would recommend that COWS
    > be "ripped out".
    > 
    > 
    > Jim
    > 
    
      Julian indicates in his message that we are not really sure what
      a good solution is and if we put something in, we are likely
      to get to a better one in the second draft. I also concur that
      the whole problem/solution space is experimental in nature, and
      ought to be treated as such.
    
      I think it is meaningless to assume that one person or another
      has a magic solution here. The problems are fairly well understood.
      One can talk about it from an algorithmic perspective (like
      error recovery) which will expose some of the assumptions/compromises
      behind this. Of course, whether that will pass muster with
      the community at large is another issue.
    > 
    > 
    > 
    > -----Original Message-----
    > From: John Hufferd [mailto:hufferd@us.ibm.com] 
    > Sent: Friday, February 01, 2002 11:31 PM
    > To: somesh_gupta@silverbacksystems.com
    > Cc: Mukund, Shridhar; ips@ece.cmu.edu
    > Subject: RE: iSCSI: No Framing
    > 
    > 
    > Somesh,
    > This is not fair, we asked for Chip Vendors that were planning on using
    > it,
    > to make themselves known and to tell us why.  Without exposing internal
    > designs, he has tried to do that, likewise Trebia, and some others have
    > said they want FIMs.  You may not want FIMs for your design, but you
    > should
    > respect these folks that have been following our spec and found it to be
    > valuable in their designs.  They are putting their money on it not just
    > talk, so we should at least give them the credit which their position
    > deserves.  Yes, of course they are disappointed with the rhetoric around
    > this issue, since they and others have been following the spec, and can
    > not
    > understand why others haven't, since they have determined it to be of
    > significant value.
    > 
    > His note does speak to the issues. (And clearly has some silly parts :-)
    > 
    > In fact he has found it of enough value to use now, since he believes
    > they
    > can not wait for iWARP to happen before the issue is addressed (and he
    > really wants iWARP).  It should be useful to listen to folks that are
    > willing to put their money where their mouth is.
    > 
    > OK, that said: don't you think we can find a way to address both your
    > position and their position, since they both seem strong and heart felt.
    > I
    > would think that the "Should Implement" or at least "May Implement"
    > would
    > give both sides enough reasonable room to meet on an even business
    > plane.
    > 
    > 
    > .
    > .
    > .
    > John L. Hufferd
    > Senior Technical Staff Member (STSM)
    > IBM/SSG San Jose Ca
    > Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
    > Home Office (408) 997-6136, Cell: (408) 499-9702
    > Internet address: hufferd@us.ibm.com
    > 
    > 
    > "Somesh Gupta" <somesh_gupta@silverbacksystems.com>@ece.cmu.edu on
    > 02/01/2002 07:10:13 PM
    > 
    > Please respond to <somesh_gupta@silverbacksystems.com>
    > 
    > Sent by:    owner-ips@ece.cmu.edu
    > 
    > 
    > To:    "Mukund, Shridhar" <Shridhar_Mukund@adaptec.com>, "'WENDT,JIM
    >        \(HP-Roseville,ex1\)'" <jim_wendt@hp.com>, <ips@ece.cmu.edu>
    > cc:
    > Subject:    RE: iSCSI: No Framing
    > 
    > 
    > 
    > Mukund,
    > 
    > With all due respect, taking on the mantle of
    > the noble engineer being denied by politically
    > motivated people, belittles the technical expertise
    > and strengths of the people who have expressed an
    > opinion against having any of the proposed schemes
    > at this time in the spec.
    > 
    > I would be very happy to work alongside a number of
    > the people on both sides of the issue (and that does
    > include you).
    > 
    > None of your arguments below refutes the technical
    > and some non-technical opinions expressed against
    > FIM.
    > 
    > Somesh
    > 
    > > -----Original Message-----
    > > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
    > > Mukund, Shridhar
    > > Sent: Friday, February 01, 2002 5:24 PM
    > > To: 'WENDT,JIM (HP-Roseville,ex1)'; ips@ece.cmu.edu
    > > Subject: RE: iSCSI: No Framing
    > >
    > >
    > > Dear Colleagues,
    > >
    > > PART 1 of 3 :
    > >
    > > The pertinent points that "I" see in Jim's email are:
    > >
    > > A. Need to hear from iSCSI chip implementers ...
    > >    The issues that you raise for e.g. in #2&#4 are simply
    > >    circumstantial( See PART 2 ). Answers to those questions
    > >    unnecessarily call for data flows and implementation
    > >    details that silicon vendors are not allowed to share
    > >    in public. While I do not know of many silicon vendors
    > >    in the multi gig space, clearly one of the competitions
    > >    I respect, namely Trebia, point to FIM as well. Granted,
    > >    no matter what, we are going to see several
    > >    trebia-on-the-other-end and adaptec-on-the-other-end
    > >    type of cost optimizations.
    > >
    > > B. The iWARP/TUF/SCTP under-current ...
    > >    The "only" message of significance in Jim's email is #5.
    > >    It seems like the iWARP/SCTP/TUF enthusiasts somehow feel
    > >    threatened that FIM-iSCSI will dilute the perceived value
    > >    proposition of iWARP :-) I for one am an enthusiast of
    > >    iWARP ideals myself, barring the proposed mechanics. I
    > >    would love to make a buck or two along with you in
    > >    building iWARP NICs, "at the right time". In the
    > >    meanwhile, iSCSI is the flag ship effort that has the
    > >    unique opportunity to make a dent in enabling IP Storage
    > >    visions. ( See PART 3 )
    > >
    > > My assertions are :
    > >     i) TUF/SCTP/iWARP is ruled out in the near term. The
    > >        mechanics are unclear as hell.
    > >     ii) FIM is the least intrusive, TCP transparent, means for
    > >        enabling low-cost(power,b/w,latency,memory,space,CPU)
    > >        RDMA transport of bulk data.
    > >     iii) I do not see significant technical reasons that
    > >        merit major changes to the iSCSI draft at this late
    > >        date.
    > >     iii) Making it OPTIONAL to use, and SHOULD only on send
    > >        provides a graceful and incremental deployment path
    > >        for "real":-) providers and users to succeed.
    > >
    > > I have personally contributed nothing to the iSCSI effort
    > > and do applaud the pains that several of you are taking to
    > > pull it all together. In that very same spirit, I respect
    > > David's decision w.r.t. consensus, whatever that turns out
    > > to be.
    > >
    > > Thanks.
    > >
    > > -Shridhar Mukund + Todd Sperry, Siva Munnangi, Dev ...
    > >
    > > ------------------------------------------------------------
    > > PART 2 of 3 : MUST delete, SHOULD read
    > >
    > > Dear Jim,
    > >
    > > Congratulations Jim! Seems like when you bowl, pins
    > > roll, unintentional as they may be. You make a "seemingly"
    > > well balanced set of arguments and "tactfully" tilt the
    > > balance towards your chosen side. I would love to be on
    > > your side ... maybe in another effort :-)
    > >
    > > I would like to introduce you to my respectable friend,
    > > who "vehemently" contradicts you on all accounts. One of
    > > his numerous quotes goes as follows:
    > > http://groups.yahoo.com/group/rdma/message/486
    > > "Much of today's usage of the Internet and IP networks
    > > is for buffer-to-buffer data transfers, often in the
    > > form of bulk data ... Gigabit and faster network transfers
    > > incur 'heavy' system resource costs, including both CPU
    > > use and system bus bandwidth, particularly on the
    > > receiving side ... The costs incurred on hosts for protocol
    > > processing and management has 'inhibited' the use of IP
    > > in the high speed bulk data domain. ... Bulk data transfer
    > > is dominated by the costs of copying and validating
    > > incoming data in order to place it at its ultimate
    > > destination."
    > >
    > > The good friend I quote above is none other than Jim
    > > Wendt himself!!! I REST MY CASE.
    > >
    > > Thanks.
    > >
    > > -Shridhar Mukund
    > >
    > > ----------------------------------------------------------------
    > > PART 3 of 3: MUST delete, OPTIONAL read
    > >
    > > On the lighter side ... since the opponents have "no" technical
    > > arguments whatsoever against FIM and it is all turning out to
    > > be a pure political gimmick, I will put on my rusting tin
    > > politician hat :-)
    > >
    > > My dear fellow iSCSI country (wo)men: If our goals are anything
    > > short of enabling the HP's, Cisco's, uSoft's, IBM's and EMC's of
    > > the world in globalization of storage for McDonald's and Macy's
    > > from Kabul to Somalia, via iSCSI, we have lost our identity!
    > > ( \insert APPLAUSE for 13 seconds )We are no more protected by
    > > the vast oceans between us and other Storage efforts. The
    > > freedom of iSCSI America is threatened from within by elements
    > > who will not blink twice when it comes to using the world's most
    > > potent BOO-bombs ... and grinnn while we end up sifting through
    > > the rubbles for iSCSI, youSCSI and theySCSI.
    > > ( \insert APPLAUSE for 11 seconds ) Beware of the elements that
    > > sleep with "our" very ideals in their privacy(burkha clad
    > > though) and yet attempt to destabilize us, only to accomplish
    > > their mutated agenda.( \insert APPLAUSE for 57 seconds )
    > >
    > > No offense folks. I respect each and every one of you and
    > > especially Jim. I think that he was only attempting to
    > > question, "are we sure ... should we commit ...".
    > > I disagree with him anyway!
    > >
    > > - the running(actually limping) mate :-) :-) :-)
    > >
    > > -----Original Message-----
    > > From: WENDT,JIM (HP-Roseville,ex1) [mailto:jim_wendt@hp.com]
    > > Sent: Tuesday, January 29, 2002 10:47 PM
    > > To: ips@ece.cmu.edu
    > > Subject: iSCSI: No Framing
    > >
    > >
    > > Perhaps we should discuss the possibility of not
    > > specifying any framing mechanism (FIM or COWS) in the
    > > first version of iSCSI.
    > >
    > > The arguments for not specifying framing include
    > > (there may be others as well):
    > > 1) The brute force approach of putting TCP receiver
    > > reassembly memory on the iSCSI NIC will work for both
    > > 1Gbps and 10Gbps. Cost is incurred for memory chips,
    > > ASIC pins, power, and board space. But, it is a
    > > feasible approach.
    > >
    > > 2) I/O bus latency (or bandwidth limitations) could
    > > mandate inbound buffering (as a 'smoothing' buffer)
    > > from the iSCSI NIC to the host system. If this buffer
    > > memory is large enough to have to be off-chip, then
    > > it will require some minimum number of memory chips
    > > to provide the necessary bandwidth, and the
    > > memory/pins/power/space costs will be incurred
    > > anyway.
    > >
    > > 3) Very large receive buffering on the NIC is only
    > > required for high-bandwidth links traversing large
    > > geographic distances (which may not be the
    > > predominate case). These specialized implementations
    > > can cost more (e.g. more memory/power/pins/etc) and
    > > customers will likely be willing to pay accordingly.
    > >
    > > 4) Many NICs will likely aspire to be combo
    > > iSCSI+TOE+Ethernet NICs allowing the host system to
    > > use a single Ethernet port for all of its network
    > > communications. The TOE function on this combo NIC
    > > will already require TCP receive buffering and off-
    > > chip buffer memory to support the existing sockets
    > > interface receive model.  More importantly, to allow
    > > senders to assume ownership of the buffer upon return
    > > from a socket send call, the TOE NIC would need to
    > > copy application send buffers into NIC memory as
    > > well.
    > >
    > > 5) The framing/marker mechanism will be an iSCSI-only
    > > solution. It is quite possible that the problem will
    > > be solved via a different, and hopefully more
    > > general, mechanism for other ULPs.
    > >
    > > 6) Both FIM and COWS are poor solutions for software
    > > implementation. COWS requires, at a minimum, the
    > > sender to read every word in the buffer. FIM either
    > > imposes additional sender gather processing (iovecs)
    > > or requires a data buffer copy (on systems that don't
    > > support HW gather on sends).
    > >
    > > 7) Unless all senders thoroughly test framing/markers
    > > now (i.e. unless the framing method is a MUST
    > > implement), there is potential for future
    > > interoperability problems as framing/marker receivers
    > > are deployed in the future.
    > >
    > > 8) Neither FIM nor COWS is an elegant solution. Are
    > > we comfortable with the legacy we are creating under
    > > the umbrella of state-of-the-art IP networked
    > > storage?
    > >
    > > 9) Is it essential to build in forward compatibility
    > > now, or will there be a different solution in the
    > > 10Gbps or 40Gbps timeframe - perhaps involving iSCSI-
    > > 2?  Will it be reasonable to update or bridge initial
    > > 1Gbps deployments?
    > >
    > >
    > > So, it would be good to hear from several iSCSI
    > > NIC/chip implementors who:
    > > - have or plan to implement FIM or COWS (or some
    > > other framing mechanism) and take advantage of it to
    > > minimize demands on on-NIC buffer memory
    > > bandwidth/quantity.
    > > - believe that all-buffers-on-chip solutions are
    > > feasible and valid (wrt the points above, including
    > > #2)
    > > - can quantify the memory/pin/power/space cost
    > > savings for all-buffers-on-chip solutions
    > >
    > > Jim
    > >
    > 
    > 
    > 
    > 
    


Home

Last updated: Tue Feb 05 14:17:55 2002
8647 messages in chronological order