SORT BY:

LIST ORDER
THREAD
AUTHOR
SUBJECT


SEARCH

IPS HOME


    [Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

    RE: iSCSI: No Framing




    Not exacly markers - those appeared in Haifa-summer of 2000 but something similar - aligned PDUs.

    Julo


    "Somesh Gupta" <somesh_gupta@silverbacksystems.com>
    Sent by: owner-ips@ece.cmu.edu

    05-02-02 21:20
    Please respond to somesh_gupta

           
            To:        John Hufferd/San Jose/IBM@IBMUS
            cc:        "Jim Pinkerton" <jpink@microsoft.com>, "Mukund, Shridhar" <Shridhar_Mukund@adaptec.com>, <ips@ece.cmu.edu>
            Subject:        RE: iSCSI: No Framing

           


    John,

    I have to point for the record, that I was the one
    that proposed markers over a year ago (not that many
    other did not also have this idea). I have also tried
    to keep this debate at a technical level. There
    are reasonable folks on both sides of the debate,
    and we should just continue in that spirit.

    Somesh

    > -----Original Message-----
    > From: John Hufferd [mailto:hufferd@us.ibm.com]
    > Sent: Tuesday, February 05, 2002 10:55 AM
    > To: somesh_gupta@silverbacksystems.com
    > Cc: Jim Pinkerton; Mukund, Shridhar; ips@ece.cmu.edu
    > Subject: RE: iSCSI: No Framing
    >
    >
    >
    > Somesh,
    > I usually give higher weight to the opinions of folks that take their
    > opinions and put their money on it, than the folks that do not.   So I am
    > not sure you can so easily dismiss the ideas of folks that are committed
    > with actual money.
    >
    > .
    > .
    > .
    > John L. Hufferd
    > Senior Technical Staff Member (STSM)
    > IBM/SSG San Jose Ca
    > Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
    > Home Office (408) 997-6136, Cell: (408) 499-9702
    > Internet address: hufferd@us.ibm.com
    >
    >
    > "Somesh Gupta" <somesh_gupta@silverbacksystems.com> on 02/05/2002 10:25:45
    > AM
    >
    > Please respond to <somesh_gupta@silverbacksystems.com>
    >
    > To:    "Jim Pinkerton" <jpink@microsoft.com>, John Hufferd/San
    >        Jose/IBM@IBMUS
    > cc:    "Mukund, Shridhar" <Shridhar_Mukund@adaptec.com>, <ips@ece.cmu.edu>
    > Subject:    RE: iSCSI: No Framing
    >
    >
    >
    > Jim,
    >
    > > -----Original Message-----
    > > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
    > > Jim Pinkerton
    > > Sent: Tuesday, February 05, 2002 6:53 AM
    > > To: John Hufferd; somesh_gupta@silverbacksystems.com
    > > Cc: Mukund, Shridhar; ips@ece.cmu.edu
    > > Subject: RE: iSCSI: No Framing
    > >
    > >
    > >
    > > In reviewing the mail alias, and after having many conversations with
    > > various folks, I would like to retract my prior opinion on ripping out
    > > markers. It appears to me as though _every_ NIC vendor is in support of
    > > markers.
    >
    >   Every NIC vendor may implement markers (or as is obvious from
    >   interoperability results, may have the capability to implement
    >   markers). On whether every (person who works at a) NIC/Chip vendor
    >   supports it from a technical perspective (i.e. whether it solves
    >   anything, or is a useful solution, or whether the analysis people
    >   are using is correct) supports it, I think the count seems to be
    >   evenly split.
    > >
    > > To me the evidence in support of keeping in "Sync and Steering with
    > > Fixed Interval Markers" is pretty compelling (now that I've been
    > > re-educated). Without going into proprietary issues - how often do NIC
    > > vendors agree unanimously on something? Also, a software implementation
    > > of markers is tractable.

    > >
    > > I have not heard a single NIC vendor in support of COWS, and I
    > > personally don't support COWS either. Thus I would recommend that COWS
    > > be "ripped out".
    > >
    > >
    > > Jim
    > >
    >
    >   Julian indicates in his message that we are not really sure what
    >   a good solution is and if we put something in, we are likely
    >   to get to a better one in the second draft. I also concur that
    >   the whole problem/solution space is experimental in nature, and
    >   ought to be treated as such.
    >
    >   I think it is meaningless to assume that one person or another
    >   has a magic solution here. The problems are fairly well understood.
    >   One can talk about it from an algorithmic perspective (like
    >   error recovery) which will expose some of the assumptions/compromises
    >   behind this. Of course, whether that will pass muster with
    >   the community at large is another issue.
    > >
    > >
    > >
    > > -----Original Message-----
    > > From: John Hufferd [mailto:hufferd@us.ibm.com]
    > > Sent: Friday, February 01, 2002 11:31 PM
    > > To: somesh_gupta@silverbacksystems.com
    > > Cc: Mukund, Shridhar; ips@ece.cmu.edu
    > > Subject: RE: iSCSI: No Framing
    > >
    > >
    > > Somesh,
    > > This is not fair, we asked for Chip Vendors that were planning on using
    > > it,
    > > to make themselves known and to tell us why.  Without exposing internal
    > > designs, he has tried to do that, likewise Trebia, and some others have
    > > said they want FIMs.  You may not want FIMs for your design, but you
    > > should
    > > respect these folks that have been following our spec and found it to be
    > > valuable in their designs.  They are putting their money on it not just
    > > talk, so we should at least give them the credit which their position
    > > deserves.  Yes, of course they are disappointed with the rhetoric around
    > > this issue, since they and others have been following the spec, and can
    > > not
    > > understand why others haven't, since they have determined it to be of
    > > significant value.
    > >
    > > His note does speak to the issues. (And clearly has some silly parts :-)
    > >
    > > In fact he has found it of enough value to use now, since he believes
    > > they
    > > can not wait for iWARP to happen before the issue is addressed (and he
    > > really wants iWARP).  It should be useful to listen to folks that are
    > > willing to put their money where their mouth is.
    > >
    > > OK, that said: don't you think we can find a way to address both your
    > > position and their position, since they both seem strong and heart felt.
    > > I
    > > would think that the "Should Implement" or at least "May Implement"
    > > would
    > > give both sides enough reasonable room to meet on an even business
    > > plane.
    > >
    > >
    > > .
    > > .
    > > .
    > > John L. Hufferd
    > > Senior Technical Staff Member (STSM)
    > > IBM/SSG San Jose Ca
    > > Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
    > > Home Office (408) 997-6136, Cell: (408) 499-9702
    > > Internet address: hufferd@us.ibm.com
    > >
    > >
    > > "Somesh Gupta" <somesh_gupta@silverbacksystems.com>@ece.cmu.edu on
    > > 02/01/2002 07:10:13 PM
    > >
    > > Please respond to <somesh_gupta@silverbacksystems.com>
    > >
    > > Sent by:    owner-ips@ece.cmu.edu
    > >
    > >
    > > To:    "Mukund, Shridhar" <Shridhar_Mukund@adaptec.com>, "'WENDT,JIM
    > >        \(HP-Roseville,ex1\)'" <jim_wendt@hp.com>, <ips@ece.cmu.edu>
    > > cc:
    > > Subject:    RE: iSCSI: No Framing
    > >
    > >
    > >
    > > Mukund,
    > >
    > > With all due respect, taking on the mantle of
    > > the noble engineer being denied by politically
    > > motivated people, belittles the technical expertise
    > > and strengths of the people who have expressed an
    > > opinion against having any of the proposed schemes
    > > at this time in the spec.
    > >
    > > I would be very happy to work alongside a number of
    > > the people on both sides of the issue (and that does
    > > include you).
    > >
    > > None of your arguments below refutes the technical
    > > and some non-technical opinions expressed against
    > > FIM.
    > >
    > > Somesh
    > >
    > > > -----Original Message-----
    > > > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
    > > > Mukund, Shridhar
    > > > Sent: Friday, February 01, 2002 5:24 PM
    > > > To: 'WENDT,JIM (HP-Roseville,ex1)'; ips@ece.cmu.edu
    > > > Subject: RE: iSCSI: No Framing
    > > >
    > > >
    > > > Dear Colleagues,
    > > >
    > > > PART 1 of 3 :
    > > >
    > > > The pertinent points that "I" see in Jim's email are:
    > > >
    > > > A. Need to hear from iSCSI chip implementers ...
    > > >    The issues that you raise for e.g. in #2&#4 are simply
    > > >    circumstantial( See PART 2 ). Answers to those questions
    > > >    unnecessarily call for data flows and implementation
    > > >    details that silicon vendors are not allowed to share
    > > >    in public. While I do not know of many silicon vendors
    > > >    in the multi gig space, clearly one of the competitions
    > > >    I respect, namely Trebia, point to FIM as well. Granted,
    > > >    no matter what, we are going to see several
    > > >    trebia-on-the-other-end and adaptec-on-the-other-end
    > > >    type of cost optimizations.
    > > >
    > > > B. The iWARP/TUF/SCTP under-current ...
    > > >    The "only" message of significance in Jim's email is #5.
    > > >    It seems like the iWARP/SCTP/TUF enthusiasts somehow feel
    > > >    threatened that FIM-iSCSI will dilute the perceived value
    > > >    proposition of iWARP :-) I for one am an enthusiast of
    > > >    iWARP ideals myself, barring the proposed mechanics. I
    > > >    would love to make a buck or two along with you in
    > > >    building iWARP NICs, "at the right time". In the
    > > >    meanwhile, iSCSI is the flag ship effort that has the
    > > >    unique opportunity to make a dent in enabling IP Storage
    > > >    visions. ( See PART 3 )
    > > >
    > > > My assertions are :
    > > >     i) TUF/SCTP/iWARP is ruled out in the near term. The
    > > >        mechanics are unclear as hell.
    > > >     ii) FIM is the least intrusive, TCP transparent, means for
    > > >        enabling low-cost(power,b/w,latency,memory,space,CPU)
    > > >        RDMA transport of bulk data.
    > > >     iii) I do not see significant technical reasons that
    > > >        merit major changes to the iSCSI draft at this late
    > > >        date.
    > > >     iii) Making it OPTIONAL to use, and SHOULD only on send
    > > >        provides a graceful and incremental deployment path
    > > >        for "real":-) providers and users to succeed.
    > > >
    > > > I have personally contributed nothing to the iSCSI effort
    > > > and do applaud the pains that several of you are taking to
    > > > pull it all together. In that very same spirit, I respect
    > > > David's decision w.r.t. consensus, whatever that turns out
    > > > to be.
    > > >
    > > > Thanks.
    > > >
    > > > -Shridhar Mukund + Todd Sperry, Siva Munnangi, Dev ...
    > > >
    > > > ------------------------------------------------------------
    > > > PART 2 of 3 : MUST delete, SHOULD read
    > > >
    > > > Dear Jim,
    > > >
    > > > Congratulations Jim! Seems like when you bowl, pins
    > > > roll, unintentional as they may be. You make a "seemingly"
    > > > well balanced set of arguments and "tactfully" tilt the
    > > > balance towards your chosen side. I would love to be on
    > > > your side ... maybe in another effort :-)
    > > >
    > > > I would like to introduce you to my respectable friend,
    > > > who "vehemently" contradicts you on all accounts. One of
    > > > his numerous quotes goes as follows:
    > > > http://groups.yahoo.com/group/rdma/message/486
    > > > "Much of today's usage of the Internet and IP networks
    > > > is for buffer-to-buffer data transfers, often in the
    > > > form of bulk data ... Gigabit and faster network transfers
    > > > incur 'heavy' system resource costs, including both CPU
    > > > use and system bus bandwidth, particularly on the
    > > > receiving side ... The costs incurred on hosts for protocol
    > > > processing and management has 'inhibited' the use of IP
    > > > in the high speed bulk data domain. ... Bulk data transfer
    > > > is dominated by the costs of copying and validating
    > > > incoming data in order to place it at its ultimate
    > > > destination."
    > > >
    > > > The good friend I quote above is none other than Jim
    > > > Wendt himself!!! I REST MY CASE.
    > > >
    > > > Thanks.
    > > >
    > > > -Shridhar Mukund
    > > >
    > > > ----------------------------------------------------------------
    > > > PART 3 of 3: MUST delete, OPTIONAL read
    > > >
    > > > On the lighter side ... since the opponents have "no" technical
    > > > arguments whatsoever against FIM and it is all turning out to
    > > > be a pure political gimmick, I will put on my rusting tin
    > > > politician hat :-)
    > > >
    > > > My dear fellow iSCSI country (wo)men: If our goals are anything
    > > > short of enabling the HP's, Cisco's, uSoft's, IBM's and EMC's of
    > > > the world in globalization of storage for McDonald's and Macy's
    > > > from Kabul to Somalia, via iSCSI, we have lost our identity!
    > > > ( \insert APPLAUSE for 13 seconds )We are no more protected by
    > > > the vast oceans between us and other Storage efforts. The
    > > > freedom of iSCSI America is threatened from within by elements
    > > > who will not blink twice when it comes to using the world's most
    > > > potent BOO-bombs ... and grinnn while we end up sifting through

    > > > the rubbles for iSCSI, youSCSI and theySCSI.
    > > > ( \insert APPLAUSE for 11 seconds ) Beware of the elements that
    > > > sleep with "our" very ideals in their privacy(burkha clad
    > > > though) and yet attempt to destabilize us, only to accomplish
    > > > their mutated agenda.( \insert APPLAUSE for 57 seconds )
    > > >
    > > > No offense folks. I respect each and every one of you and
    > > > especially Jim. I think that he was only attempting to
    > > > question, "are we sure ... should we commit ...".
    > > > I disagree with him anyway!
    > > >
    > > > - the running(actually limping) mate :-) :-) :-)
    > > >
    > > > -----Original Message-----
    > > > From: WENDT,JIM (HP-Roseville,ex1) [mailto:jim_wendt@hp.com]
    > > > Sent: Tuesday, January 29, 2002 10:47 PM
    > > > To: ips@ece.cmu.edu
    > > > Subject: iSCSI: No Framing
    > > >
    > > >
    > > > Perhaps we should discuss the possibility of not
    > > > specifying any framing mechanism (FIM or COWS) in the
    > > > first version of iSCSI.
    > > >
    > > > The arguments for not specifying framing include
    > > > (there may be others as well):
    > > > 1) The brute force approach of putting TCP receiver
    > > > reassembly memory on the iSCSI NIC will work for both
    > > > 1Gbps and 10Gbps. Cost is incurred for memory chips,
    > > > ASIC pins, power, and board space. But, it is a
    > > > feasible approach.
    > > >
    > > > 2) I/O bus latency (or bandwidth limitations) could
    > > > mandate inbound buffering (as a 'smoothing' buffer)
    > > > from the iSCSI NIC to the host system. If this buffer
    > > > memory is large enough to have to be off-chip, then
    > > > it will require some minimum number of memory chips
    > > > to provide the necessary bandwidth, and the
    > > > memory/pins/power/space costs will be incurred
    > > > anyway.
    > > >
    > > > 3) Very large receive buffering on the NIC is only
    > > > required for high-bandwidth links traversing large
    > > > geographic distances (which may not be the
    > > > predominate case). These specialized implementations
    > > > can cost more (e.g. more memory/power/pins/etc) and
    > > > customers will likely be willing to pay accordingly.
    > > >
    > > > 4) Many NICs will likely aspire to be combo
    > > > iSCSI+TOE+Ethernet NICs allowing the host system to
    > > > use a single Ethernet port for all of its network
    > > > communications. The TOE function on this combo NIC
    > > > will already require TCP receive buffering and off-
    > > > chip buffer memory to support the existing sockets
    > > > interface receive model.  More importantly, to allow
    > > > senders to assume ownership of the buffer upon return
    > > > from a socket send call, the TOE NIC would need to
    > > > copy application send buffers into NIC memory as
    > > > well.
    > > >
    > > > 5) The framing/marker mechanism will be an iSCSI-only
    > > > solution. It is quite possible that the problem will
    > > > be solved via a different, and hopefully more
    > > > general, mechanism for other ULPs.
    > > >
    > > > 6) Both FIM and COWS are poor solutions for software
    > > > implementation. COWS requires, at a minimum, the
    > > > sender to read every word in the buffer. FIM either
    > > > imposes additional sender gather processing (iovecs)
    > > > or requires a data buffer copy (on systems that don't
    > > > support HW gather on sends).
    > > >
    > > > 7) Unless all senders thoroughly test framing/markers
    > > > now (i.e. unless the framing method is a MUST
    > > > implement), there is potential for future
    > > > interoperability problems as framing/marker receivers
    > > > are deployed in the future.
    > > >
    > > > 8) Neither FIM nor COWS is an elegant solution. Are
    > > > we comfortable with the legacy we are creating under
    > > > the umbrella of state-of-the-art IP networked
    > > > storage?
    > > >
    > > > 9) Is it essential to build in forward compatibility
    > > > now, or will there be a different solution in the
    > > > 10Gbps or 40Gbps timeframe - perhaps involving iSCSI-
    > > > 2?  Will it be reasonable to update or bridge initial
    > > > 1Gbps deployments?
    > > >
    > > >
    > > > So, it would be good to hear from several iSCSI
    > > > NIC/chip implementors who:
    > > > - have or plan to implement FIM or COWS (or some
    > > > other framing mechanism) and take advantage of it to
    > > > minimize demands on on-NIC buffer memory
    > > > bandwidth/quantity.
    > > > - believe that all-buffers-on-chip solutions are
    > > > feasible and valid (wrt the points above, including
    > > > #2)
    > > > - can quantify the memory/pin/power/space cost
    > > > savings for all-buffers-on-chip solutions
    > > >
    > > > Jim
    > > >
    > >
    > >
    > >
    > >
    >
    >
    >
    >





Home

Last updated: Tue Feb 05 21:17:59 2002
8659 messages in chronological order