RE: iSCSI : iSCSI parameter default values

["Somesh Gupta" <somesh_gupta@silverbacksystems.com>]

Comments in Line.

> -----Original Message-----
> From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
> Robert Snively
> Sent: Tuesday, October 02, 2001 9:40 AM
> To: 'John Hufferd'; Robert Snively
> Cc: ips@ece.cmu.edu
> Subject: RE: iSCSI : iSCSI parameter default values
>
>
> John,
>
> What I really wanted to point out is that the destination
> of the data is NOT the buffer, nor is it a memory space
> behind the buffer.  It is a non-volatile protected storage medium,
> which may be either a set of disk drives with the proper degrees
> of RAID redundancy or (at least temporarily) some kind of
> replicated non-volatile RAM.
>
> It is that protected space which must be made available for received data.
> If you tack the data into the command PDU, you must treat the
> whole combination.  You may treat it as protected, or you may treat
> it as recoverable, depending on how complex you want your PDU
> retransmission algorithm to be, but in either case you will have
> to post-process it to separate the command, interpret the command,
> and move the data portion to the proper resting place, where it
> can be available for subsequent accesses by other commands.
>
> If you leave it separate, you can treat the command as a unit
> which does not have to be separated out or analyzed until you get
> around to it, placing the data in a pre-allocated portion of
> the non-volatile storage for later tagging and referencing when
> the command is interpreted.
>
> Better yet, if you use R2T, you can request the data at the
> proper time to have it delivered (in any convenient order of
> course) directly to the proper destination disk drive or to a pre-tagged
> portion of non-volatile storage.  It is this type of operation that
> is truly "zero-copy" in the SCSI and storage sense.

  The caveat should be "the way it is done today". with the
  proper PDU header/data split, and
  with the right buffer management scheme, zero copy can
  be done with immediate/unsolicited data.

>
> Bob Snively                        e-mail:    rsnively@brocade.com
> Brocade Communications Systems     phone:  408 487 8135
> 1745 Technology Drive
> San Jose, CA 95110
>
>
>
> > -----Original Message-----
> > From: John Hufferd [mailto:hufferd@us.ibm.com]
> > Sent: Sunday, September 30, 2001 2:29 AM
> > To: Robert Snively
> > Cc: ips@ece.cmu.edu
> > Subject: RE: iscsi : iscsi parameter default values
> >
> >
> >
> > Robert,
> > You only have to perform moves, if you have not coded your
> > buffer manager
> > to manage Chained Segments.  Since this is a very old and highly used
> > technique in buffer management, especially involving
> > Telecommunications, I
> > am surprised that you have not considered it.  Perhaps I am missing
> > something in what you are trying to say but I have used it
> > often to avoid
> > moving data and obtaining zero-copy.  It is also works well
> > with devices
> > that support Scatter Gather, which is the usual case with Disk.
> >
> > Immediate data is simple, perhaps the simplest of all our functions to
> > code.  You are correct, that it means that there may have to
> > be reasonably
> > sized buffers.  This argument was valid years ago, but it is
> > now just too
> > hard not to have large buffers in a storage controller (in
> > fact now small
> > cost more).
> >
> > Yes, I know there are limits to what I said above, but that is why the
> > ImmediateData=No can be used.  But your focus and arguments are old
> > arguments, for devices that thought memory was too expensive
> > to use a great
> > deal of it.
> >
> > Example: since most uses of ImmediateData will probably
> > average less then
> > 4k (example Database pages),  four thousand connections means
> > 16 Meg, I do
> > not think you can even get a decent price on memories that
> > small.  If you
> > have a device that supports 4 thousand host connections, I
> > suggest that 16
> > meg is not an issue.  Why would we even discuss it?
> >
> >
> > .
> > .
> > .
> > John L. Hufferd
> > Senior Technical Staff Member (STSM)
> > IBM/SSG San Jose Ca
> > Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
> > Home Office (408) 997-6136
> > Internet address: hufferd@us.ibm.com
> >
> >
> > Robert Snively <rsnively@Brocade.COM>@ece.cmu.edu on
> > 09/28/2001 04:31:12 PM
> >
> > Sent by:  owner-ips@ece.cmu.edu
> >
> >
> > To:   Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu
> > cc:
> > Subject:  RE: iscsi : iscsi parameter default values
> >
> >
> >
> > Julian,
> >
> > For SCSI Write operations, ImmediateData = yes is
> > the most demanding mode for the target.  If I understand
> > it correctly, it essentially over-rides the R2T state,
> > allowing the first burst to be included as immediate data.
> > In SCSI, except for special cases like this, the target
> > is the device in charge of data transfers.
> >
> > This mode requires the target to have a guaranteed collection of
> > received buffer space adequate to receive all possible
> > outbound queued operations from all possible initiators
> > through all possible target sessions (which may sum to
> > 1000s of output operations), regardless of what other
> > buffer-intensive operations may be going on in the device.
> >
> > It then requires the device to provide association with each
> > of those commands instead of just the commands it has elected
> > to activate.  Without immediate data, the command buffer can be
> > separated and separately managed from the data buffer,
> > limiting buffer requirements.
> >
> > It then requires the device to operate in a non-zero-copy
> > mode (which raises buffer utilization and increases latency)
> > while the device analyzes the command to determine what should
> > be done with the data.  It further requires the subsequent
> > data (if there is more than one PDU to be transmitted) to
> > be separately managed, and perhaps actually stored in a
> > separate operation if the buffer must be cleared to make
> > space available for it.  It further requires the target to
> > analyze the data for completeness before going on to determine
> > what to do with it.
> >
> > Sure, it is easy for the initiator, but so is everything else
> > except out-of-order reads.
> > It is the target you are stressing with this.
> >
> > For local sub-LANs, and depending on the actual buffering
> > available in the target, the performance may actually be lower with
> > ImmediateData allowed, because zero copy behavior of the
> > target to non-volatile media is not available, which raises
> > buffer utilization.
> >
> > Have I missed something that would change my mind?
> >
> > Bob
> >
> >
> > > -----Original Message-----
> > > From: Julian Satran [mailto:Julian_Satran@il.ibm.com]
> > > Sent: Friday, September 28, 2001 10:38 AM
> > > To: ips@ece.cmu.edu
> > > Subject: RE: iscsi : iscsi parameter default values
> > >
> > >
> > >
> > >
> > > Robert,
> > >
> > > I disagree that Immediate is "the most demanding" mode.  That
> > > is not my
> > > experience and not what I heard from most of the developers.  On the
> > > contrary immediate data do not require a separate indexing
> > > operation on the
> > > target (as non-immediate unsolicited do) and that was the
> > base for the
> > > consensus to have them negotiated separately.
> > >
> > > For the software initiator it is the most inexpensive way to
> > > perform short
> > > data transfers (4-8k) typical for major applications like
> > > Database access
> > > and so it is for lightweight target.
> > >
> > > Julo
> > >
> > >
> > >
> > >
> > >
> > >                     Robert Snively
> > >
> > >                     <rsnively@broc       To:     John
> > > Hufferd/San Jose/IBM@IBMUS, Julian
> > >                     ade.com>
> > > Satran/Haifa/IBM@IBMIL
> > >                                          cc:
> > > ips@ece.cmu.edu
> > >                     28-09-01 17:25       Subject:     RE:
> > > iscsi : iscsi parameter default values
> > >                     Please respond
> > >
> > >                     to Robert
> > >
> > >                     Snively
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > I vote no as the default value on ImmediateData.
> > >
> > > A default value of yes on ImmediateData requires implementation
> > > of the most complex and demanding mode of operation as the
> > > default.  SCSI has traditionally made its default behavior the
> > > simplest and most encompassing, setting more sophisticated
> > > behavior by subsequent agreement.  While it may be your earnest
> > > desire to encourage the implementation of this function, it
> > > is not appropriate to demand that as the default behavior
> > > of all devices.  In particular, there is no special benefit
> > > to providing ImmediateData in low-cost local sub-lans.
> > >
> > > If you want to encourage it in a profile, fine, but demanding
> > > it as the default in the core standard is not appropriate.
> > >
> > > Note that the behavior of SCSI is traditionally managed
> > > entirely by the target.  As such, there has never before now
> > > been a requirement for the target to, as a default, accept
> > > any PDU except a command or task management function
> > > that was not explicitly solicited.  That is one of the mechanisms
> > > that assists SCSI in achieving a low-overhead zero copy
> > > capability while operating with a large number of initiators
> > > and with deep command queues.
> > >
> > > Bob Snively                        e-mail:    rsnively@brocade.com
> > > Brocade Communications Systems     phone:  408 487 8135
> > > 1745 Technology Drive
> > > San Jose, CA 95110
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: John Hufferd [mailto:hufferd@us.ibm.com]
> > > > Sent: Friday, September 28, 2001 1:33 AM
> > > > To: Julian Satran
> > > > Cc: ips@ece.cmu.edu
> > > > Subject: RE: iscsi : iscsi parameter default values
> > > >
> > > >
> > > >
> > > > I also agree with this.  It should be yes.
> > > >
> > > > .
> > > > .
> > > > .
> > > > John L. Hufferd
> > > > Senior Technical Staff Member (STSM)
> > > > IBM/SSG San Jose Ca
> > > > Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
> > > > Home Office (408) 997-6136
> > > > Internet address: hufferd@us.ibm.com
> > > >
> > > >
> > > > Julian Satran/Haifa/IBM@IBMIL@ece.cmu.edu on 09/27/2001
> > 09:50:21 AM
> > > >
> > > > Sent by:  owner-ips@ece.cmu.edu
> > > >
> > > >
> > > > To:   ips@ece.cmu.edu
> > > > cc:
> > > > Subject:  RE: iscsi : iscsi parameter default values
> > > >
> > > >
> > > >
> > > >
> > > > The one that I have trouble living with is ImmediateData.
> > > > This is important
> > > > for low-end desktops and hardly matters for large boxes.
> > > > As such I would suggest it stays as yes.
> > > >
> > > > Julo
> > > >
> > > >
> > > >
> > > >                     "Eddy
> > > >                     Quicksall"            To:     "'Santosh Rao'"
> > > > <santoshr@cup.hp.com>,
> > > >                     <EQuicksall@med        <ips@ece.cmu.edu>
> > > >                     iaone.net>            cc:
> > > >                     Sent by:              Subject:     RE:
> > > > iscsi : iscsi
> > > > parameter default values
> > > >                     owner-ips@ece.c
> > > >                     mu.edu
> > > >
> > > >
> > > >                     27-09-01 17:22
> > > >                     Please respond
> > > >                     to "Eddy
> > > >                     Quicksall"
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > I like your defaults below.
> > > >
> > > > But, section 5 says:
> > > >
> > > >  The initial Login request MUST include the InitiatorName and
> > > >  SessionType key=value pairs.
> > > >
> > > > Since SessionType is REQUIRED, naming a default would imply a
> > > > possible typo
> > > > in the spec.
> > > >
> > > > Eddy
> > > >
> > > > -----Original Message-----
> > > > From: Santosh Rao [mailto:santoshr@cup.hp.com]
> > > > Sent: Wednesday, September 26, 2001 10:29 PM
> > > > To: ips@ece.cmu.edu
> > > > Subject: iscsi : iscsi parameter default values
> > > >
> > > >
> > > > All,
> > > >
> > > > With the issue of mode page vs. login keys having [almost]
> > > drawn to a
> > > > close, I would like to
> > > > raise the below issues again, on the subject of default
> > > > values for login
> > > > keys and other iscsi
> > > > parameters :
> > > >
> > > >
> > > >    * In keeping with traditional use of scsi mode pages,
> > > iscsi should
> > > > not specify any default
> > > >      settings for any mode pages that continue to exist for iscsi.
> > > > "Default values" for mode
> > > >      pages are target specific and should not be bound to
> > > the protocol
> > > > draft.
> > > >
> > > >    * MORE IMPORTANTLY, I read the default for EMDP as
> > being set to 1
> > > > :-<  This implies that
> > > >      targets must be always prepared to deal with out of
> > > > order data and
> > > > initiators must be
> > > >      prepared to deal with out of order data, unless the initiator
> > > > performs a mode select to
> > > >      disable it. [which defeats all the previous advantages
> > > > gained thru
> > > > mandating use of login
> > > >      keys for other negotiations.]. A default, if it were
> > to exist,
> > > > should be 0. (in-order, by
> > > >      default).
> > > >
> > > >    * Conservative specification of defaults for login
> > keys along the
> > > > following lines :
> > > >                             MaxConnections = 1
> > > >                             FMarker = "no"
> > > >                             InitialR2T = "yes"
> > > >                             BidiInitialR2T = "yes"
> > > >                             ImmediateData = "no"
> > > >                             DataPDULength = 16
> > > >                             MaxOutstandingR2T = 1
> > > >                             DataPDUInOrder = "yes"
> > > >                             ErrorRecoveryLevel = 0
> > > >                             SessionType = "normal"
> > > >
> > > >    * Should the iscsi protocol require a "Lun Control Mode
> > > Page"? IOW,
> > > > is an EnableCRN bit
> > > >      required at the transport layer ? If the device server
> > > capability
> > > > is to be negotiated , I
> > > >      suggest this bit be moved to a SCSI ULP Mode Page such as the
> > > > "Control Mode Page", through a
> > > >      T10 change as a part of the SCSI changes being
> > driven by iscsi.
> > > >
> > > > Comments ?
> > > >
> > > > Thanks,
> > > > Santosh
> > > >
> > > >
> > > > Santosh Rao wrote:
> > > >
> > > > > There are the separate issues of :
> > > > >
> > > > >    * iscsi's specification of defaults for its mode pages
> > > > which is not
> > > > in line with mode page
> > > > >      usage. This impacts the target's ability to enforce
> > > > values other
> > > > than the protocol
> > > > >      specified default, if the initiator were to not use mode
> > > > sense/select.
> > > > >
> > > > >    * default settings for login keys.
> > > > >
> > > > >    * Is there a need for the "LUN Control Mode Page" and whether
> > > > "Enable CRN" should be in a
> > > > >      transport specific mode page ?
> > > > >
> > > > > which need to be driven to closure as well.
> > > > >
> > > > > Regards,
> > > > > Santosh
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
>