RE: iSCSI: Negotiation clarifications still needed

[Bill Studenmund <wrstuden@wasabisystems.com>]

On Fri, 24 May 2002 pat_thaler@agilent.com wrote:

** note there's a proposed experiment at the end. Please try it and see
what you think of how your code does :-) **

> Martins,
>
> Comments referenced by the same items Martins used.
>
> 1. Julian sent an email saying he would put the text
> I proposed in (though the text you quoted is not the
> whole text).
>
> 2. I think that the principle we have been using on
> text negotiation was that each key negotion is a
> separate item. Your proposal would be counter to that
> and I don't think it would be an improvement. The
> target should be allowed to respond to any complete
> key-value pair it has received. When a key-value
> pair is straddling the PDU bondary, then it shouldn't
> respond to that key until the complete key-value pair
> has been received.

Has anyone actually operated negotiations in the mode where keys don't
fully fit and there are multiple keys to deal with?

I'd expect not since the default PDU data payload is 8k, and I don't think
we really have 8k worth of negotiation parameters. While we do have
security parameters which can get over 8k, the security negotiations I saw
(when scanning the spec) only had one key in flight at a time (thus we
don't get into this case).

To be honest, what Martins proposed is exactly what I read the spec to
indicate! :-) It's also what I've coded, and was pretty easy to do.

The example showing the spec text negotiation certainly indicated that's
the way to do things. :-|

The reason I really like what Martins has suggested (and which we talked
about in private EMail) is that my head hurts when I think about how to
build code to deal with what you describe.

Here's the case that makes my head hurt:

Initiator builds up a set of keys and starts to negotiate them. Due to the
fact it had too much text, it can't send it all in one go. So it builds up
the first PDU, and sends it. It now 1) must remember where it is (so it
can finish in the next PDU), and 2) deal with the key negotiations that
come back from the target in response to the first PDU.

When the reply comes back from the target, the initiator has both handle
the keys the target sends back and the unsent ones it has. You partially
cover that below by talking about not letting the target initiate keys
when we had a partial send. But there's another case too:

Say the response from the target ALSO doesn't fit in one PDU. Yes, the
target's not going to respond to the key=value pair that wasn't complete,
but say its response to everything else was bigger than the initiator's
representation of said keys. For this case to happen, the target's
response has to grow more than the first bit of the PDU-spanning key/value
pair.

The case where I can see this happening is we had a mix of boolean-or keys
and other keys. And for these bolean-or keys, the initiator wanted NO but
the target wanted YES. So the target's response is bounded to be bigger.
Or maybe there were some Rejects in there when the offer was less than 7
bytes long.

So now we have TWO PDU-spanning key sets. And the target didn't originate
anything. :-)

Yes, this is a rare case. But it can happen. And, for iSCSI to be robust,
we need to prevent it.

When I think about this, my head spins.

It seems much simpler to me to let one side finish sending before the
other starts responding. That way we won't end up with having to deal with
any corner cases. :-)

> There is one potential corner case issue that should
> to be covered. Targets can initiate keys. If key-value
> pairs didn't straddle PDU boundaries, then ensuring
> that there is a clear originator for each offered key
> is easy. You can originate any key that you haven't
> received an offer of from your partner. But now keys
> can straddle PDUs. If the text between the last separater
> and the end of the PDU is Ma, then you don't know what
> key your partner has started to offer. If the partner
> was starting to offer MaxBurstSize and you offer it in
> the next PDU of the exchange, both sides may think they
> are originator.

Yeah, that's the other corner case.

What Martins suggests just seems easier to me. We decouple negotiation
parsing from sending/receiving negotiation packets. The only time we hand
data to the nego. parser, all the key/value pairs are there. We then
generate all the response we want to send. If needed we send said over
multiple PDUs.

Among other things, we DON'T need to even deal with split keys at all;
they get reassembled before we look at them!

Here's a sample logic flow:

dealing_with_input_login_cmd/rsp()
{
    if (have_more_to_send_from_last_time) {
	send a PDU containing more data
	if (this_PDU_finishes_data_flow)
		clear have_more_to_send_from_last_time;
	return;
    }

    if (PDU.len != 0) {
	read PDU's data to buffer
	if (this_pdu_isn't_last)
		return
    }

    Process_input_buffer_and_generate_response

    Send a PDU containing data
    if (not_all_data_sent)
	set have_more_to_send_from_last_time;
}

The same logic flow works for both text and login, and avoids LOTS of
headaches.

*** Experiement ***

Here's a simple experiment to try. Try cranking down the default PDU
length your implementation uses to something rediculously small, like 64
bytes, and see what happens when you negotiate a login. Are you happy with
the results?

Take care,

Bill