RE: iSCSI: Iterating long text responses

["Rod Harrison" <rod.harrison@windriver.com>]

Mark,

	It was more of a general concern for the iterative process
than thoughts of a specific non-idempotent text command. If
text commands are ever used in an IOCTL like capacity we
might run into problems, but as you say there are probably
better ways to do that sort of thing, like Mode Select.

	I would still prefer to have the initiator able to pace the
receipt of, and prematurely end, large text responses but I
understand this may be too expensive for some targets.

	Requiring the initiator to allocate an arbitrarily large
buffer for #5 is a little concerning, especially since it
seems that could be larger than the negotiated max PDU size.
For that reason I would prefer option #4 over #5, but I
think we'll need to add a sequence number to the text
response to allow the initiator to detect out of order PDUs.

	- Rod

-----Original Message-----
From: mbakke@cisco.com [mailto:mbakke@cisco.com]
Sent: Thursday, July 05, 2001 3:11 PM
To: Rod Harrison
Cc: IPS
Subject: Re: iSCSI: Iterating long text responses



Rod-

You are correct that this assumes that the initiator can do
the same text command multiple times.  I think that the real
question is whether attempting to allow non-idempotent text
commands, and providing easier mechanisms for doing them is
a requirement.

If so, your addition to #3 would work.

If not, it would place unnecessary burden on the target.

At the moment, I can't think of much that would need to be
added to iSCSI from a (non-negotiation) text command point
of view.  In fact, we are supposed to be careful to limit
what can be done in this way, since most other functionality
we can envision should be left to other standard protocols.

Any thoughts on specific uses for non-idempotent text
commands?

--
Mark

Some more comments:

Rod Harrison wrote:
>
>         We have a concern over option #5, the outline
below assumes
> that it is safe to issue the same text command more than
> once with no side effects. This does seem safe for the
> current use of SendTargets, however it might not be if
this
> mechanism is extended for other long response text
commands.

True; SendTargets doesn't care.  However, we seem to have a
general consensus that the use of text commands should be
carefully limited; what are the real chances that some
functionality comes along that we really need, and that does
not fit in?  I don't see that really happening.

>         Our preference would be for option #3 with #4 a
close
> second. Number 4 has the disadvantage that the initiator
has
> to deal with a stream of unsolicited text responses with
no
> idea of when they might stop.

Actually, an initiator can do the same thing with #4 as with
#5;
it could issue the text command, and just keep track of the
number
of bytes of response that it was unable to keep.  It could
then
allocate enough space, and re-do the request.  #5 is just a
cleaner
mechanism to do this.

At any rate, the burden is on the initiator as to whether it
wants to make use of these mechanisms.

>         Perhaps #3 with the following modifications might
suffice.
>
>         Add an offset to the initiator R2T (iR2T?)
allowing the
> target to choose to either buffer the whole response or
> regenerate it on the fly. The target should know whether
the
> data can be regenerated safely and so can make this
> decision.
>
>         Add a final bit to the iR2T allowing the initiator
to stop
> the response transfer at any time. Mandate that the
> initiator must either receive all the data or send an iR2T
> with final bit set to indicate no more data is required.
> This prevents a target getting stuck with the response
data
> in a buffer forever.
>
>         A typical transfer might be.
>
>         I->T Text command (F bit set)
>         T->I Text response (F bit clear)
>         I->T iR2T incl. valid offset and length (F bit
clear)
>         T->I Text response (F bit clear)
>
>         ....
>
>         Sequence either ends with
>
>         T->I Text response (F bit set)
>
>         or
>
>         I->T iR2T Offset 0, length 0, (F bit set)
>
>         Since the first text response is unsolicited all
current
> text command implementations can remain unchanged. Only if
> the target responds with the F bit clear does the
initiator
> need to change.


>         - Rod
>
> -----Original Message-----
> From: owner-ips@ece.cmu.edu
[mailto:owner-ips@ece.cmu.edu]On
> Behalf Of
> Mark Bakke
> Sent: Friday, June 29, 2001 9:18 PM
> To: IPS
> Subject: iSCSI: Iterating long text responses
>
> Initiator developers-
>
>    Please respond to the questions at the end.
>
> Thanks,
>
> Mark
>
> The current iSCSI draft allows text command and response
> PDUs of up to 4096 bytes.  While we don't see any real
> problems for the command PDU size, commands such as
> SendTargets
> can easily exceed the response size.
>
> There are several ways we can fix this.  The first two
> solutions
> require no differences in the current iSCSI text command
and
> response; the latter three involve the use of the F bit.
>
> 1. Assume that such commands are done on a "special"
> connection
>    or are handled completely in software, and allow its
> response
>    PDU to be as large as it needs to be.
>
>    This one appears too restrictive to be a solid
solution.
> It
>    will also weaken any data digests done on the longer
PDU.
>
> 2. Create an iterative SendTargets key (and do the same
for
> any
>    other text commands that need this), that would allow
the
>    initiator to request the "next target" or "next
address".
>
>    This would work, but would require any new command that
> needed
>    a large response to implement an iterator.  It also
means
> that
>    each part of the response from the iterator would have
to
> fit
>    in the 4k PDU.
>
> The remainder of these require that only one text command
> sequence
> be outstanding on a connection at a given time.  They use
> the F
> bit to indicate the last PDU of the sequence.  Note that
> connection
> allegiance is assumed for the entire sequence, and text
> commands
> are never retried on another connection; a new command is
> issued
> instead.
>
> 3. Do a text-response R2T, where the initiator controls
when
> the
>    next partial response is sent.  This would be more
> generic:
>
>    I->T  Text Command (F bit set)
>    T->I  Text Response (first PDU, F bit cleared)
>    I->T  Text Command (with some indicator that we want
> more)
>    T->I  Text Response (next PDU, F bit cleared)
>
>    ...
>
>    I->T  Text Command (with indicator that we want more)
>    T->I  Text Response (last PDU, F bit set)
>
>    The main problem with this is that the target must keep
> track
>    of the state of its responses; if the initiator just
> stops asking,
>    it may have to keep a buffered response around until it
> times out,
>    the connection is dropped, or another text command
comes
> along.
>
> 4. Allow multiple response PDUs to a single text command:
>
>    I->T  Text Command (F bit set)
>    T->I  Text Response (first PDU, F bit cleared)
>    T->I  Text Response (next PDU, F bit cleared)
>
>    ...
>
>    T->I  Text Response (last PDU, F bit set)
>
>    Basically, we are doing (3) without the R2T.  The
> initiator,
>    upon sending the text command, must be prepared either
to
>    accept as many PDUs as come back, or throw them away if
> it
>    can't handle them.  This solution is a lot like #1, but
> with
>    the response spread across 4k PDUs.
>
>    Also note that this (and the following scheme) avoid
the
> problem
>    of the target keeping state; it sends ALL of the
response
> PDUs
>    without the initiator asking for them.
>
> 5. Do #4, but allow the initiator to specify the amount of
> data
>    it is willing to accept:
>
>    I->T  Text Command (F bit set,
AcceptResponseLength=4096)
>    T->I  Text Response (first PDU, F bit set,
> TotalResponseLength=12288)
>
>    In the above example, we have created a new text
command
> field:
>
>       AcceptResponseLength
>
>    And in the text response PDU:
>
>       TotalResponseLength
>
>    The initiator indicated it was willing to accept a 4k
> response.
>    The target returned the first 4k in the text response,
> but set
>    the F bit since it was at its limit.  It also returned
a
>    TotalResponseLength field.  Since this was greater than
> the
>    AcceptResponseLength, the initiator knows the amount of
> buffer
>    space it will need to get the full response.  It can
then
> choose
>    whether it will re-send the command, and if so, can
give
> it a
>    large enough response length:
>
>    I->T  Text Command (F bit set,
> AcceptResponseLength=12288)
>    T->I  Text Response (first PDU, F bit clear)
>    T->I  Text Response (next PDU, F bit clear)
>    T->I  Text Response (last PDU, F bit set,
> TotalResponseLength=12288)
>
>    Note that most initiators will probably send an
> AcceptResponseLength
>    of the largest response they would normally accept, and
> that most
>    target responses will fit in one or a few PDUs anyway.
>
>    #5 is really a compromise between #3 and #4; the target
> has the
>    benefit of being less statefull, and the initiator has
> the benefit
>    of controlling the amount of data it receives.
>
> I would like to recommend either #4 or #5.  I think that
#5
> is
> probably the safest solution, but #4 may not be a problem
> for anyone.
>
> Assuming that none of the implementors of initiators have
a
> problem
> with #4, I would pick that.
>
> If they do have a problem with it, we should go with #5.
>
> Targets probably don't care much whether we do #4 or #5.
>
> Initiator developers-
>
>   Please indicate which solution (#4 or #5) appeals to
you.
>
> --
> Mark A. Bakke
> Cisco Systems
> mbakke@cisco.com
> 763.398.1054

--
Mark A. Bakke
Cisco Systems
mbakke@cisco.com
763.398.1054