RE: iSCSI: Negotiation clarifications still needed

[Martins Krikis <mkrikis@yahoo.com>]

--- pat_thaler@agilent.com wrote:

> Martins,
> 
> Comments referenced by the same items Martins used.
> 
> 1. Julian sent an email saying he would put the text
> 
> I proposed in (though the text you quoted is not the
> 
> whole text).

Julian put in part of the text (in 12-92). I quoted 
the part that he didn't put in. I'm wondering why.
Julian, I asked you once personally, and asked on
the list again. Why don't you put the remaining part
of Pat's text in so that there are no doubts
about this issue?

> 2. I think that the principle we have been using on 
> text negotiation was that each key negotion is a 
> separate item. Your proposal would be counter to
> that 
> and I don't think it would be an improvement.

I thought so too until yesterday, in fact my
implementation was NOT sending just blank PDUs,
but I realized it had a subtle bug, having to do
with what you describe as a "corner case" below.

> The
> target should be allowed to respond to any complete
> key-value pair it has received. When a key-value
> pair is straddling the PDU bondary, then it
> shouldn't
> respond to that key until the complete key-value
> pair
> has been received.

of course, since it may not even know for sure
what the straddling key or value is...

> There is one potential corner case issue that should
> to be covered. Targets can initiate keys. If
> key-value
> pairs didn't straddle PDU boundaries, then ensuring 
> that there is a clear originator for each offered
> key
> is easy. You can originate any key that you haven't 
> received an offer of from your partner.

That's roughly why I made the proposal to look for
a "data-end" flag.

> But now keys
> can straddle PDUs. If the text between the last
> separater
> and the end of the PDU is Ma, then you don't know
> what
> key your partner has started to offer. If the
> partner 
> was starting to offer MaxBurstSize and you offer it
> in
> the next PDU of the exchange, both sides may think
> they 
> are originator.

And that is bad. Also, if we don't consider
MaxBurstSize as offered, then the side that put 
"Ma" in the PDU is in a somewhat uneasy situation for
that key's state... it cannot consider it sent,
however, presumably it had some value ready for
it, so it had gone through some logic for that key.
It just becomes a clumsy situation.

> I suggest one of the following
> a) don't allow keys to straddle PDU boundaries.

This would work, but is not as clean as having
a data-end flag introduced, which signals that
now the other side should respond. 

With your proposal the Originator
when it prepares each key and value must check
that the key and '=' fits entirely, otherwise
it must put it back and likely go through the
process again in the next round. Here we have
the key + value generation logic tightly coupled
with the iSCSI encapsulator (which knows how much
is fitting).

> b) don't allow originating a key when the last login
> PDU ended in a partial key.

Do you mean don't allow originating any key at
all or do you mean the same as in (c) below?
If the former, then this is practically the
same as what I'm proposing, except my restriction
would be in effect if any part of key=value pair
is straddled. That requires just checking for NUL-s
at the end of the PDU and not looking farther back 
for '='-s. Still an even cleaner aproach is
a new flag in the header.

> c) don't allow offering a key where the start of the
> key
> matches a partial key at the end of the last login 
> PDU.

This would work too. But is not a very clean
aproach to the problem. Before considering
whether to originate a key we have to compare it
to the incomplete key that's left over from the
just received PDU. Furthermore, the side that
sent the incomplete key ideally should remember
that it sent an incomplete key and check incoming
keys against the incomplete one to catch the other
side in cheating...

My point really is that ideally we want to see
something like this:

Initiator:

  key1=value1
  key2=value2
  key3=value3
  ...
  key17=value17

Target:

  key1=value1
  ...
  key17=value17
  key18=value18
  ...
  key34=key34

Initiator:
  
  key18=value18
  ...
  key34=key34

Done.

We don't want to worry about how this splits up
in PDUs. Just like when I'm writing to a BSD-socket
I don't care in which IP datagram each byte travels.
And if I'm trying to send a certain block of data
and expect the other side to answer once it receives
it all, I don't like it interrupting my thought
and beginning to answer little by little after
each IP packet received. 

The point is that negotiation would be more natural
if it where "block-based", where each side can
send its block of data to the other side without
being interrupted with partial answers and new
keys requiring answers. Right now we've made PDU 
the block, but that is not convenient because PDU
size is limited, and thus not all data (that a
node may have worked hard preparing) may fit.
Because it all doesn't fit, the other side may
start originating the same data that we would
have sent if only the PDU had been larger. We
get into extra complications trying to avoid
the problem of both sides thinking that they
originated something.

Perhaps an even better analogy is the IP protocol
itself. IP datagrams can get fragmented en route.
Yet I know of no protocols that would work on
individual fragments of them, they all first
assemble the whole IP datagram, then see what
can be done with it. Same thing here. An iSCSI
node likely has a bunch of key=value pairs
that it wants to send to the other side and it
is an extra hassle to keep checking how many 
will fit and how many won't, then adjusting
state for those, etc., etc. It is way cleaner
to just pass all data down to the encapsulating
layer, knowing that the other side won't do a thing
before all this data hasn't been received and
reassembled there.

I'm saying my proposal is cleaner. Your proposals
use bandwidth more efficiently but are more complex.
This is not a performance critical area of the
specification, nor are we even worrying about a
very likely case. I honestly believe that typically
most everything that a node wants to send will fit.
Thus, I think the cleaner aproach should be preferred.

> 3. Yes, we noticed a little while ago that losing a

This was my #4, BTW, but that's OK.

> packet at the end of negotiation could hang things
> up
> though the concern is mainly for a full feature
> phase
> negotiation. Looking at 6.8, any timeout during
> negotiation causes the login and its TCP connection
> to
> be terminated. The whole negotiation process (see
> the 
> point about origninators in 2) depends upon a
> one-by-one
> exchange of PDUs. PDU loss has to terminate it. 

I'm beginning to skip and snip text here because
I was explicitly worried about the Text 
Request/Response case, not Logins.

> The concern was Full Feature Phase negotation. Until
> 
> negotiation ends, it can be reset and no values
> change.
> When the target sends the last Text Response PDU,
> then
> it thinks negotiation has ended and it applies the
> new values. 

So you're suggesting that the target commits to the
new values as soon as it sends the last Text
Response PDU. This is what I've been doing, but from
other posts on the list I know that other people
preferred to wait till the next PDU from the
initiator.

> If that PDU doesn't reach the
> initiatior,
> then it terminates the entire negotiation and

just to be precise, it = initiator.

> continues
> to use the old values. The two ends are using
> different
> values.

Yes, that's what I'm worried about and don't see
a good way out. If the workgroup is aware that
this is possible and is happy to live with it, 
then I can probably leave the issue alone.

> We decided to not raise this as an issue because it
> is
> such a corner case - we are operating over a
> reliable
> connection so PDUs shouldn't be lost (unless the
> whole
> path goes down in which case it doesn't matter).
> Also,
> there are few values exchanged during full feature
> so
> it isn't worthwhile to add complexity. 

OK, I'll take your word for it.

Thanks,

  Martins Krikis, Intel Corp.

Disclaimer: these opinions are mine and may not
            be those of my employer



__________________________________________________
Do You Yahoo!?
LAUNCH - Your Yahoo! Music Experience
http://launch.yahoo.com