Review of the 07 draft

["Robert Griswold" <rgriswold@Crossroads.com>]

Julian:

First, great work on this initiative and please forgive my huge email.  

We have undertaken a review of the 07 version, and even though
Crossroads will be at the meetings in London, we thought it would be
prudent to post these observations on the current specification.  If we
have completely misread the document, or have missed resolutions that
have come across the reflector, we apologize up front.  I have also sent
you some editorial only comments for your review.

++++++++++

Section 1.2.2.1 (and others):
The third paragraph up from the bottom of page 19 refers to the action
that a target should take when receiving commands outside of a range,
that are not correctly marked as immediate, with the right flags set.
The question is in regards to the MUST instruction on "silently ignore".
Assuming the I_T nexus is established, the devices are communicating and
the target receives what could only correctly be termed a protocol error
from the initiator it has established communication with, why would the
target not respond with an error back?  If in fact the error is
transport related, and the initiator really did send the correct PDU,
then the target being mute on the reception of the this command leaves
the initiator guessing.  There are other points in the specification
where the target is instructed to "silently ignore."  

Section 1.2.2.2:
The fifth paragraph indicates that a "large difference" between StatSN
and ExpStatSN may indicate a problem, but the number required by the
specification as the maximum before an initiator must take action is 2.1
Billion!  This seems like a very big number, and most implementers would
want to take action long before this limit was reached.  What is the
thinking behind this number?  Is there something we aren't
understanding?

Section 1.2.5:
On page 25, the second full paragraph indicates that the definition of
the use of target tags is not specifically specified by the protocol.
While we see possible uses for Target Transfer Tags, we would like to
understand if the author's feel that limiting the interpretation of its
use to that described in 2.17.4 would be helpful.  Clearly this 32-bit
field plays little role in the overall usefulness iSCSI command
protocol, so someone must have felt that it would have been a great idea
to have this in the specification.  Can that be described?

Section 1.2.6
This section covers the topic of connection termination, which is also
covered in the discussion of the Logout command in section 2.14.  There
seems to be some mingling of the concept of TCP logout with the concepts
around termination of a communication channel for storage protocols.
The concept of sending a logout, then waiting for the receiving end to
do some clean-up, flushing, and or additional communication seems odd to
the command.  The Logout directive should require no further
communication, and should put neither initiator nor target into the
state of expecting any such communication.  We would like to see the
Logout command (not the request for a logout) be free of any
expectations once sent, or received.  It seems logical that since the
initiator is the responsible party for sending this command (we would
also like to begin some discussion on a target only Logout command, like
SRP is working on now) that it would have done all of the required
cleanup and preparation before issuing the command.  Implementation of
target code to make decisions on what outstanding commands it should or
shouldn't give some credence to before breaking the connection seems out
of place. 

Section 1.2.7
On page 28, in the fourth to last paragraph in this section, there is a
description on how a target can assume an iSCSI Initiator Name, as
learned through authentication.  This is a confusing paragraph, and we
cannot determine if it is implying that the target should use the
initiator name it has been given through some service, or that it was
authorized to use by an initiator that discovered its iSCSI functions.
Can this paragraph be further expanded to clarify how the name was
determined?

Section 1.2.9.1
The last paragraph on this section has as 42 words in it's first
sentence, with the word "message" being used six times.  Suggested
rewording:
"Relying solely on the "message length" information from the iSCSI
message header may make it impossible to find iSCSI message boundaries
in subsequent TCP segments; due to the loss of a TCP segment containing
the iSCSI message length."

Section 1.2.9.2
The second full paragraph's last two sentences have odd wording.
Suggested rewording:
"In practice, though, iSCSI is not expected to handle infinitely long
streams; stream addressing will wrap around at 2**32-1."

The first sentence of the next paragraph is difficult to interpret.
Suggested rewording:
"This model assumes that the iSCSI layer will deliver complete PDUs to
underlying layers in single (atomic) operations."
You may also want to consider adding words that explain the reasoning
behind this assumption, as some who don't care about those layers may
not understand why it would be a reasonable assumption.

The rest of this section uses the word MUST a few times in the
description of actions or implementation assumptions.  While the use of
the word MUST may make complete sense to the topic, and the points they
are being applied to, it seems out of place for an optional layer of the
model.  If the Synch and Steering layer is truly optional, then how can
the specification make claims to what MUST happen?

Section 1.5.3
The second paragraph on page 39 uses the term nexuses, but does not
clearly build the relationship that the ISID rule is attempting to
explain.  Can the level (I_T, I_T_L, I_T_L_x) of nexus be identified
that is being referred to here, to help clarify the ISID rule mentioned
in the previous paragraph?

Section 2.2.2
The definition of the AHS (see below, 2.2.3.1) seems to be covered as a
global PDU property, but only op-code 0x01 (SCSI Command) seems to make
use of it.  Since the AHS seems to be only needed to encapsulate an
extended CDB, then why define its possible existence for all PDUs?  Can
the AHS be used in other commands?

Section 2.2.2.4
This section defines the behavior of non-specific fields in op-code use,
but not completely.  One portion of the field is explained (P/F bit),
but the use of the rest of the field is left to the reader to find in
the other op-codes.  Would it make more sense to pull the definition of
the P/F bit out of this section, then simply tell the reader that the
other bits are defined in the op-code specific sections?  Maybe tabulate
the P/F usage in this section, to the op-code specific use?

Section 2.2.2.7
The second sentence of this paragraph defines that the LUN field can be
used for "some other purpose."  While there may be a good reason for
overloading this field, we would like to see some more information on
the nature of this use.  What about inserting "opcode-specific" in from
of the word "purpose".

Section 2.2.3.1
The inclusion of a bit that identifies if the CDB is extended should not
live in an optional header.  Expecting that all implementations would
correctly code the use of an optional header to return critical data
that should be identified with the required portion of the CDB seems out
of place.  We would suggest that the use of a reserved bit or a bit in a
reserved byte be used, from the BHS, be used to identify an extended
CDB.  Putting the spillover of the CDB into the AHS (section 2.3.6) is
not an issue, but flagging if there is spillover should not be in the
AHS.

Section 2.4.3
This section has seen a lot of discussion, and we agree with the
position that Rob Elliot and David Black took on the disassociation of
the iSCSI responses from the ASC/ASCQ SCSI status codes.  While the
discussion of ACA and it's relationship to iSCSI may not be resolved
quickly, there needs to be a way to avoid polluting the expectation of
the SAM status to the devices with the iSCSI response codes.

Section 2.5.1
This section seems to build on the work done in FCP for out of CDB
delivery of task management.  But the behavior of a target on receipt of
a Target Cold Reset should be similar to that of a Logout (assuming
behavior as explained above in comments on section 1.2.6), where the
target returns as little information as possible, and does not attempt
to clean-up or respond upon reception of the command.  Also, this is the
only section in the document where the term MIB is used, and since
management of iSCSI is outside of the scope of this document, we would
prefer to see the last sentence in the paragraph on Target Cold Reset
removed.

Section 2.8
While the section on Bookmarks is brand new, and will probably go
through more definition, the double definition of this field should be
removed.  If it stays as Bookmark, and it is optional, then it is not
reserved.

Section 2.8.5
This section allows (in the first paragraph) large binary items to be
encoded as hex or decimal.  This discussion is ongoing now, and we
believe the encoding should be hex.

The third to last sentence in this section alludes to the ability to use
key=value pairs to perform active operations.  Since iSCSI active
operations are interpreted by us as related to SCSI operations, what is
being said here?  Does this mean that an initiator could perform
mode-parameter or task management operations inside of Text Messages?
If so, then more definition of this behavior is needed.

Section 2.9.5
The second paragraph attempts to instruct the reader to order response
key=value pairs in the same order they were received, but provides
little direction on why, or what possible consequences would be if it
were not done in that order.  Why is this sentence needed, except for
instruction on best practices?

Section 2.10.1
The instructions to targets on supporting multiple connections run up
against the minimum requirement of targets to support only one
connection.  If this section is a better reason to require at least two
connections at a minimum, then the specification should call that the
minimum.  If initiators are going to expect to be able to use this
command, but targets that only support one connection would refuse it,
than what good is this instruction going to be?  Would a target designer
who develops an iSCSI target create the ability to open two connections
to support only this situation, and not expand the interface to run two
connections in Full Feature Mode?

Section 2.12.4
If the limit of reflected data is 4096 bytes, then the instruction to
the initiator should not be "avoid sending", it should be "MUST NOT"
send more than 4096 bytes.  Is there a reason why the initiator would
want to send more than those number of bytes?

Section 2.14
This section was discussed above in relationship to section 1.2.6.  The
Logout command should be simple and immediate, with no further work
needed by the target then the clearing of internal buffers, and the
reporting on the next Login (CHECK CONDITION) that the Logout was
performed. 

Section 2.16
The last sentence in this section seems at odds with good error
detection for targets, and at odds with section 2.19.1.  If a SNACK
received has and error, whether or not it was a miss-directed PDU, or a
transit error, why would a target just silently ignore it?  Should it
use the SNACK Rejected code of the Reject response, or at least tell the
initiator something?  Besides, the use of the word "must" seems
inconsistent with what this sentence is trying to say.  If there is a
good reason for a target to ignore this, then it should be mentioned,
and the directive should be "MUST" if the expectations of the initiator
are set that way.

Section 2.16.1
The last paragraph of this section is confusing.  We cannot determine
how it should be rewritten, since the intent is not clear.  Why would
the target issue a message requesting Logout simply because it cannot
process as SNACK that is optional anyway?  How does the initiator know
if the target supports in-connection or out-of-connection recovery with
regards to what the target does after discarding the SNACK?

Section 2.17.3
What is the purpose of the last sentence on page 87?  It seems to
indicate that there is a good reason (besides the obvious) why the
Desired Data Length should not be zero, but falls short of explaining,
or directing the reader with a "MUST," or "SHOULD."

Section 2.17.4
This topic was discussed above in section 1.2.5.  

Section 3
We would like to see the explanation in this section clarified with
regards to the use of mode parameter changes for SCSI settings, versus
iSCSI settings.  Suggested rewording for the first sentence of the
second paragraph:
"The mode parameters for iSCSI behavior cannot be set by SCSI mode-set
but can be retrieved by SCSI mode-sense commands."
Suggested additional wording:
"SCSI mode parameters (non-iSCSI specific) cannot be set or retrieved
from within iSCSI mode page commands."  -  Or something like that.

Section 4.2
Under the key SecurityContextComplete=yes, there is a description of
what the party sending this key should do, finishing with the phrase
"until the handshake is complete."  What is the scenario for avoiding a
lock condition in this handshake?  Should the party use some standard
timeout? Where is this defined?

Section 7
The second paragraph in this section defines an assumption of the
specification.  Potential manufacturers of iSCSI targets will want to
know under what circumstances will the saving of sense and status
information differ than those already done for FC or SCSI operation;
specifically with regards to iSCSI Logout and iSCSI task management
commands.  Can this paragraph have more details added?

Section 7.1c
This description of the use of the Retry bit seems to put the onus of
interpretation of the Retry bit on the target for command replay.  Is it
at all possible for a Replay request to get issued, within the scope of
an initiator delaying its acknowledgement, such that the command
actually did get completed, but the Retry bit was set too soon?  Would
one assume that an initiator just avoids doing something like this?  

Section 7.6
This section uses the term "ULP timeout", but there is no definition or
reference to where it can be found.

Section 7.9
By recommending (in the first paragraph) that a connection be tested in
HA environments with iSCSI NOP-ping command, aren't the authors
recommending that designers use bandwidth consuming methods for
determining this?  If you assume a multiple path connection (fabric) for
such iSCSI implementations, then couldn't a bunch of initiators
potentially flood the fabric with such commands to "recognize a failing
connection?"

Section 7.11.1 and 7.11.2
Specifically, what time-outs are being referred to here?  The time a
target or initiator should or might wait for acknowledgement from the
other node that error recovery is being attempted?

Section 7.11.3
This section seems to not agree with the methods for completing a Logout
command as discussed earlier.  Item number (2) in this section indicates
that the initiator should treat the reception of an Asynchronous Message
requesting a Logout command to that of a TCP failure, yet the target is
instructed (section 2.14) to treat the reception of a Logout command as
an opportunity to complete any commands "it deems necessary."  Again,
the need for a Target Logout and Initiator Logout seems apparent.

Thanks,
Bob

Robert Griswold
Technologist
Crossroads Systems, Inc.
512-928-7272