Re: iSCSI: Iterating long text responses

[Mark Bakke <mbakke@cisco.com>]

Rod-

I don't see where option #4 is any better than option #5,
since 5 is the same as 4, but with the initiator able to
restrict the amount of data returned.  The response is still
sent back in multiple text response PDUs, just as in #4.

I don't think that out-of-order PDUs are a problem, since
the text command would have connection allegiance.  I think
that we should explicitly state that text commands are not
to be recovered if something goes wrong.  That would eliminate
the need for a text response sequence number.

--
Mark

Rod Harrison wrote:
> 
> Mark,
> 
>         It was more of a general concern for the iterative process
> than thoughts of a specific non-idempotent text command. If
> text commands are ever used in an IOCTL like capacity we
> might run into problems, but as you say there are probably
> better ways to do that sort of thing, like Mode Select.
> 
>         I would still prefer to have the initiator able to pace the
> receipt of, and prematurely end, large text responses but I
> understand this may be too expensive for some targets.
> 
>         Requiring the initiator to allocate an arbitrarily large
> buffer for #5 is a little concerning, especially since it
> seems that could be larger than the negotiated max PDU size.
> For that reason I would prefer option #4 over #5, but I
> think we'll need to add a sequence number to the text
> response to allow the initiator to detect out of order PDUs.
> 
>         - Rod
> 
> -----Original Message-----
> From: mbakke@cisco.com [mailto:mbakke@cisco.com]
> Sent: Thursday, July 05, 2001 3:11 PM
> To: Rod Harrison
> Cc: IPS
> Subject: Re: iSCSI: Iterating long text responses
> 
> Rod-
> 
> You are correct that this assumes that the initiator can do
> the same text command multiple times.  I think that the real
> question is whether attempting to allow non-idempotent text
> commands, and providing easier mechanisms for doing them is
> a requirement.
> 
> If so, your addition to #3 would work.
> 
> If not, it would place unnecessary burden on the target.
> 
> At the moment, I can't think of much that would need to be
> added to iSCSI from a (non-negotiation) text command point
> of view.  In fact, we are supposed to be careful to limit
> what can be done in this way, since most other functionality
> we can envision should be left to other standard protocols.
> 
> Any thoughts on specific uses for non-idempotent text
> commands?
> 
> --
> Mark
> 
> Some more comments:
> 
> Rod Harrison wrote:
> >
> >         We have a concern over option #5, the outline
> below assumes
> > that it is safe to issue the same text command more than
> > once with no side effects. This does seem safe for the
> > current use of SendTargets, however it might not be if
> this
> > mechanism is extended for other long response text
> commands.
> 
> True; SendTargets doesn't care.  However, we seem to have a
> general consensus that the use of text commands should be
> carefully limited; what are the real chances that some
> functionality comes along that we really need, and that does
> not fit in?  I don't see that really happening.
> 
> >         Our preference would be for option #3 with #4 a
> close
> > second. Number 4 has the disadvantage that the initiator
> has
> > to deal with a stream of unsolicited text responses with
> no
> > idea of when they might stop.
> 
> Actually, an initiator can do the same thing with #4 as with
> #5;
> it could issue the text command, and just keep track of the
> number
> of bytes of response that it was unable to keep.  It could
> then
> allocate enough space, and re-do the request.  #5 is just a
> cleaner
> mechanism to do this.
> 
> At any rate, the burden is on the initiator as to whether it
> wants to make use of these mechanisms.
> 
> >         Perhaps #3 with the following modifications might
> suffice.
> >
> >         Add an offset to the initiator R2T (iR2T?)
> allowing the
> > target to choose to either buffer the whole response or
> > regenerate it on the fly. The target should know whether
> the
> > data can be regenerated safely and so can make this
> > decision.
> >
> >         Add a final bit to the iR2T allowing the initiator
> to stop
> > the response transfer at any time. Mandate that the
> > initiator must either receive all the data or send an iR2T
> > with final bit set to indicate no more data is required.
> > This prevents a target getting stuck with the response
> data
> > in a buffer forever.
> >
> >         A typical transfer might be.
> >
> >         I->T Text command (F bit set)
> >         T->I Text response (F bit clear)
> >         I->T iR2T incl. valid offset and length (F bit
> clear)
> >         T->I Text response (F bit clear)
> >
> >         ....
> >
> >         Sequence either ends with
> >
> >         T->I Text response (F bit set)
> >
> >         or
> >
> >         I->T iR2T Offset 0, length 0, (F bit set)
> >
> >         Since the first text response is unsolicited all
> current
> > text command implementations can remain unchanged. Only if
> > the target responds with the F bit clear does the
> initiator
> > need to change.
> 
> >         - Rod
> >
> > -----Original Message-----
> > From: owner-ips@ece.cmu.edu
> [mailto:owner-ips@ece.cmu.edu]On
> > Behalf Of
> > Mark Bakke
> > Sent: Friday, June 29, 2001 9:18 PM
> > To: IPS
> > Subject: iSCSI: Iterating long text responses
> >
> > Initiator developers-
> >
> >    Please respond to the questions at the end.
> >
> > Thanks,
> >
> > Mark
> >
> > The current iSCSI draft allows text command and response
> > PDUs of up to 4096 bytes.  While we don't see any real
> > problems for the command PDU size, commands such as
> > SendTargets
> > can easily exceed the response size.
> >
> > There are several ways we can fix this.  The first two
> > solutions
> > require no differences in the current iSCSI text command
> and
> > response; the latter three involve the use of the F bit.
> >
> > 1. Assume that such commands are done on a "special"
> > connection
> >    or are handled completely in software, and allow its
> > response
> >    PDU to be as large as it needs to be.
> >
> >    This one appears too restrictive to be a solid
> solution.
> > It
> >    will also weaken any data digests done on the longer
> PDU.
> >
> > 2. Create an iterative SendTargets key (and do the same
> for
> > any
> >    other text commands that need this), that would allow
> the
> >    initiator to request the "next target" or "next
> address".
> >
> >    This would work, but would require any new command that
> > needed
> >    a large response to implement an iterator.  It also
> means
> > that
> >    each part of the response from the iterator would have
> to
> > fit
> >    in the 4k PDU.
> >
> > The remainder of these require that only one text command
> > sequence
> > be outstanding on a connection at a given time.  They use
> > the F
> > bit to indicate the last PDU of the sequence.  Note that
> > connection
> > allegiance is assumed for the entire sequence, and text
> > commands
> > are never retried on another connection; a new command is
> > issued
> > instead.
> >
> > 3. Do a text-response R2T, where the initiator controls
> when
> > the
> >    next partial response is sent.  This would be more
> > generic:
> >
> >    I->T  Text Command (F bit set)
> >    T->I  Text Response (first PDU, F bit cleared)
> >    I->T  Text Command (with some indicator that we want
> > more)
> >    T->I  Text Response (next PDU, F bit cleared)
> >
> >    ...
> >
> >    I->T  Text Command (with indicator that we want more)
> >    T->I  Text Response (last PDU, F bit set)
> >
> >    The main problem with this is that the target must keep
> > track
> >    of the state of its responses; if the initiator just
> > stops asking,
> >    it may have to keep a buffered response around until it
> > times out,
> >    the connection is dropped, or another text command
> comes
> > along.
> >
> > 4. Allow multiple response PDUs to a single text command:
> >
> >    I->T  Text Command (F bit set)
> >    T->I  Text Response (first PDU, F bit cleared)
> >    T->I  Text Response (next PDU, F bit cleared)
> >
> >    ...
> >
> >    T->I  Text Response (last PDU, F bit set)
> >
> >    Basically, we are doing (3) without the R2T.  The
> > initiator,
> >    upon sending the text command, must be prepared either
> to
> >    accept as many PDUs as come back, or throw them away if
> > it
> >    can't handle them.  This solution is a lot like #1, but
> > with
> >    the response spread across 4k PDUs.
> >
> >    Also note that this (and the following scheme) avoid
> the
> > problem
> >    of the target keeping state; it sends ALL of the
> response
> > PDUs
> >    without the initiator asking for them.
> >
> > 5. Do #4, but allow the initiator to specify the amount of
> > data
> >    it is willing to accept:
> >
> >    I->T  Text Command (F bit set,
> AcceptResponseLength=4096)
> >    T->I  Text Response (first PDU, F bit set,
> > TotalResponseLength=12288)
> >
> >    In the above example, we have created a new text
> command
> > field:
> >
> >       AcceptResponseLength
> >
> >    And in the text response PDU:
> >
> >       TotalResponseLength
> >
> >    The initiator indicated it was willing to accept a 4k
> > response.
> >    The target returned the first 4k in the text response,
> > but set
> >    the F bit since it was at its limit.  It also returned
> a
> >    TotalResponseLength field.  Since this was greater than
> > the
> >    AcceptResponseLength, the initiator knows the amount of
> > buffer
> >    space it will need to get the full response.  It can
> then
> > choose
> >    whether it will re-send the command, and if so, can
> give
> > it a
> >    large enough response length:
> >
> >    I->T  Text Command (F bit set,
> > AcceptResponseLength=12288)
> >    T->I  Text Response (first PDU, F bit clear)
> >    T->I  Text Response (next PDU, F bit clear)
> >    T->I  Text Response (last PDU, F bit set,
> > TotalResponseLength=12288)
> >
> >    Note that most initiators will probably send an
> > AcceptResponseLength
> >    of the largest response they would normally accept, and
> > that most
> >    target responses will fit in one or a few PDUs anyway.
> >
> >    #5 is really a compromise between #3 and #4; the target
> > has the
> >    benefit of being less statefull, and the initiator has
> > the benefit
> >    of controlling the amount of data it receives.
> >
> > I would like to recommend either #4 or #5.  I think that
> #5
> > is
> > probably the safest solution, but #4 may not be a problem
> > for anyone.
> >
> > Assuming that none of the implementors of initiators have
> a
> > problem
> > with #4, I would pick that.
> >
> > If they do have a problem with it, we should go with #5.
> >
> > Targets probably don't care much whether we do #4 or #5.
> >
> > Initiator developers-
> >
> >   Please indicate which solution (#4 or #5) appeals to
> you.
> >
> > --
> > Mark A. Bakke
> > Cisco Systems
> > mbakke@cisco.com
> > 763.398.1054
> 
> --
> Mark A. Bakke
> Cisco Systems
> mbakke@cisco.com
> 763.398.1054

-- 
Mark A. Bakke
Cisco Systems
mbakke@cisco.com
763.398.1054