RE: iSCSI: Negotiation clarifications still needed

["Julian Satran" <Julian_Satran@il.ibm.com>]

Pat your proposed 2b may be what we are looking for - i.e. a responder may not originate a key if it has an incomplete key text.

The text we may want to add to section 4.2 is:

Key=value pairs may span PDU boundaries. A responder having a received partial key text MUST refrain from originating any new key=value negotiations until it has no incomplete key text. This way one avoids having both negotiating entities assuming the originator role in a negotiation.


Julo

pat_thaler@agilent.com 05/25/2002 12:16 AM Please respond to pat_thaler

To:        mkrikis@yahoo.com, Julian Satran/Haifa/IBM@IBMIL         cc:        ips@ece.cmu.edu         Subject:        RE: iSCSI: Negotiation clarifications still needed

Martins,

Comments referenced by the same items Martins used.

1. Julian sent an email saying he would put the text 
I proposed in (though the text you quoted is not the 
whole text).

2. I think that the principle we have been using on 
text negotiation was that each key negotion is a 
separate item. Your proposal would be counter to that 
and I don't think it would be an improvement. The
target should be allowed to respond to any complete
key-value pair it has received. When a key-value
pair is straddling the PDU bondary, then it shouldn't
respond to that key until the complete key-value pair
has been received.

There is one potential corner case issue that should
to be covered. Targets can initiate keys. If key-value
pairs didn't straddle PDU boundaries, then ensuring 
that there is a clear originator for each offered key
is easy. You can originate any key that you haven't 
received an offer of from your partner. But now keys
can straddle PDUs. If the text between the last separater
and the end of the PDU is Ma, then you don't know what
key your partner has started to offer. If the partner 
was starting to offer MaxBurstSize and you offer it in
the next PDU of the exchange, both sides may think they 
are originator.

I suggest one of the following
a) don't allow keys to straddle PDU boundaries.
b) don't allow originating a key when the last login 
PDU ended in a partial key.
c) don't allow offering a key where the start of the key
matches a partial key at the end of the last login 
PDU.

3. Yes, we noticed a little while ago that losing a
packet at the end of negotiation could hang things up
though the concern is mainly for a full feature phase
negotiation. Looking at 6.8, any timeout during
negotiation causes the login and its TCP connection to
be terminated. The whole negotiation process (see the 
point about origninators in 2) depends upon a one-by-one
exchange of PDUs. PDU loss has to terminate it. 

Therefore, the target commits to the end of login
as described for T5 target. It has sent the final 
login response with a status of zero. Moves to S5.
If the login response doesn't get to the initiator,
then either the initiator will close the connection
due to the timeout. Since the target is in S4, loss 
of the transport connection will cause it to go to
S8 and R1 of the cleanup state machine. It presumably
will not take the M2 transition because the intiator
isn't going to do cleanup for a connection it thinks
wasn't in full feature phase. It will take M1 due to
timeout - not elegant but good enough.

The concern was Full Feature Phase negotation. Until 
negotiation ends, it can be reset and no values change.
When the target sends the last Text Response PDU, then
it thinks negotiation has ended and it applies the
new values. If that PDU doesn't reach the initiatior,
then it terminates the entire negotiation and continues
to use the old values. The two ends are using different
values.

We decided to not raise this as an issue because it is
such a corner case - we are operating over a reliable
connection so PDUs shouldn't be lost (unless the whole
path goes down in which case it doesn't matter). Also,
there are few values exchanged during full feature so
it isn't worthwhile to add complexity. 

Regards,
Pat


-----Original Message-----
From: Martins Krikis [mailto:mkrikis@yahoo.com]
Sent: Friday, May 24, 2002 12:39 PM
To: Julian Satran
Cc: ips@ece.cmu.edu
Subject: iSCSI: Negotiation clarifications still needed


The previous thread went on too long, but since it
has now quieted down, I'll conjecture that the
following are the aspects that still need to be
addressed.

1. Not everybody seemed to have noticed that it is
   NOT legal to send the same key again, if it
   has once been negotiated (including negotiations
   that end with a reserved value (Reject, 
   Irrelevant or NotUnderstood).

I think it would benefit the draft to add the
sentence that Pat proposed to paragraph 5 or page
72 in 12-92: "Sending the key again would be a
re-negotiation". That I think would make it crystal
clear.

2. When the key=value pairs that Originator is
   sending are broken across multiple PDUs, it
   is not clear whether the Responder may start
   responding to the keys as soon as it receives
   them or whether it should send blank PDUs back
   (as in the example on page 164 of 12-92) until
   it gets a PDU, the data part of which ends in
   a NUL byte (thus signaling that there are no
   broken key=value pairs at the end of it).

I am proposing that the draft should make it explicit
that only blank PDUs are to be sent. This allows
decoupling of key=value generation from
their encapsulation in PDUs (i.e., the generating
logic need not worry about whether a key=value
pair will fit and go out in this PDU or has to
be retained to go out in the next). I can explain
in detail why this is important (it has to do
with teh possibility of receiving the "just-about
outgoing" keys) but I'm keeping this "brief".

Furthermore, it is my feeling that instead of
checking the last bytes of a PDU for NUL, it
would be better if the end-of-data was marked by
a flag in the header. This way encapsulation will
be simpler---just put as much data in the PDU as
fits there and raise the flag if it isn't all,
instead of checking whether it ends in a NUL and
possibly shortening data to make it not end like
that.

3. There is an opinion that on page 73 of 12-92,
   the phrase that says "or the responder may
   select an admissible value" is in contradiction
   to the very next sentence. There is also an
   opinion that this phrase is entirely unnecessary
   and detrimental to achieving broad 
   interoperability (I call it "cutting slack to
   misbehaving or incompatible originators").

I don't have a suggestion since I consider the
"feature" that this phrase allows of little 
importance to a properly built iSCSI node.

4. This is new. When doing Text Request/Response
negotiations (i.e., in FFP), it seems 
that the Initiator commits to the new values
when it receives a response from the Target with
the F bit set. It is unclear when the Target should
commit. Should it switch to using the new values
as soon as it sends its response with the F-bit
set, or should it do so only when it knows that
the Initiator received its response? 

Commiting right away is simpler and since responses
with F-bits set have TTT=0xffffffff and thus
may not be reset, sounds plausible. If the
values have importance on the next reception,
it may also be important to commit timely. However,
what if the Initiator doesn't get this response? 
Target now has committed, Initiator hasn't. Committing
later puts the burden on Initiator to send something
effectively telling "I've received your final
response". Otherwise the Target will time out and
not commit. This response can get lost too.

Basically, it is beginning to look a bit like
(what was it called?) "distributed consensus problem"?
I think it goes like this:

Two generals that are on oposite sides of the
enemy want to synchronize their attack, and
start sending messengers through with messages
like 
  "attack at dawn->", 
  "<-ok, attack at dawn",
  "I know you know we attack at dawn->",
  "<-, I know you know I know we attack at dawn",
  etc., etc., ...
But at no point can they commit yet...

Is anybody else worried about this?
Anyway, so when should a target commit? Page 83
of 12-92 is the relevant reference.

Thanks,

  Martins Krikis

Disclaimer: these opinions are my own and may
            not be those of my employer.


__________________________________________________
Do You Yahoo!?
LAUNCH - Your Yahoo! Music Experience
http://launch.yahoo.com