Re: iSCSI : Review Feedback on iSCSI Rev 06

[Stephen Bailey <steph@cs.uchicago.edu>]

Santosh,

> What I was pointing out was that this upper limit needs to be lower than
> DataPDULength, failing which fragmentation can occur.

I see what you're saying now, but I don't agree that the iSCSI spec
requires this fragmentation.  Data PDUs are not NOP, Text or
Login[Response] PDUs.  The DataPDULength limit applies only to data
PDU length (or, we should change the name to PDULength).

The most important point to make is that both the initiator and the
target be able to control the total size of any PDU sequence received
(therefore, sent) during an operational session.

If we want to punt completely on iterators for SendTarget response and
use transport-level flow control to handle an arbitrary amount of data
(i.e. no iterator for SendTarget), so be it.  However, it is essential
that if we do this, SendTarget be prohibited during operational
sessions.

Outside of a SendTarget interaction, I do not see any way that an
iSCSI-specified text interaction would result in a gush larger than
EITHER 4K OR the minimum DataPDULength of 512 bytes (not that this
matters, as mentioned above).  However, we do allow extended
interactions with the X-*, and in order to ensure that these extended
interactions function properly across implementations, we need to set
a bound on the resources that the transport implementation must
provide for these interactions.

> This is the way the FC-GS-2/FC-GS-3 GID_FT command operates and it
> allows for a faster target discovery than the iterative use of other
> FC-GS-2/GS-3 name server queries such as GA_NXT (GAN).

Do people actually use GID_FT?  It doesn't seem wise.  I always used
GA_NXT, and it seemed that all the other FCP implementors I asked did
as well (it's been a while).

The problem is precisely analogous to what you see FCP.  Data from an
operation like GID_FT is going to go into buffers which are allocated
locally to the FCP driver.  This pool is going to be sized based upon
operational assumptions (e.g. # of outstanding commands), and if you
ask the name server for the entire name list at once, you're probably
going to consume all those buffers and engage link-layer flow control
before you can wade through the results.  In a fabric, this can clog
everything up pretty good.

Fortunately, in the iSCSI/TCP case, engaging flow control like this
will not block the channel, so it's actually marginally better to do
this in iSCSI than in FCP.  As long as you are not performing storage
operations, AND the target is somewhat sophisiticated about returning
the data (big IF---it must not consume all it's local resources trying
to SEND the target list data which is going to drizzle out slowly
under TCP flow control), everybody will get along just fine.

This underscores another argument for iterators.  For large targets,
simply SENDING the entire target list at once might be a very resource
intensive operation.  You could suggest that the target just send a
little at a time, and that's sorta OK, except that the initiator has
to set some timeout on receiving the response, and if the targets are
allowed to drizzle the data at an arbitrary rate, it's hard to chose a
timeout value.

I can argue both sides of this (I have, right?), but given that
nobody besides me seems to believe in iterators, we at least have to
make sure that the big gush approach works properly.

Steph