RE: iSCSI: Negotiation clarifications still needed

[Martins Krikis <mkrikis@yahoo.com>]

> It's to avoid corner cases while doing what you say above, "preparing 
> its
> strings and chopping them into PDUs."
> 
> The simplest thing in my mind is to:
> 
> 1) read the other side's offers into a buffer
> 
> 2) parse that buffer generating responses to another buffer
> 
> 3) send the output buffer
> 
> Does that make sense? You don't have to agree, just does it make sense?
> 
> The problem I see with what Pat described is that if you are in
> operational negotiation and have more than 1 PDU worth of data, you 
> will
> be receiving input while you're in step 3. You have to be able to 
> re-enter
> steps 1 & 2 before you can finish step 3.
> 
> <PAT> I think that is over stating the complexity. During negotiation,
> the other side can only have one PDU outstanding at a time. That is 
> explictly
> stated for TextNegotiation PDUs. For LoginPDUs, it isn't explicit but 
> since
> they are immediate and the initiator can only count on the target 
> having one
> immediate buffer for the connection it can't send another LoginRequest 
> until
> it gets the LoginResponse without risking that a PDU will be dropped. 
> Also
> allowing anything other than one oustanding LoginPDU at a time would 
> open 
> the possibility that both sides offered the same key. Therefore, a 
> target
> won't get the next PDU until after it sends the response PDU (or 
> vice-versa
> for an initiator). A PDU won't come in while a PDU is being sent from 
> the
> outgoing negotiation buffer.

Pat, I must interrupt you here. (A subliminal part of this exercise
is to show how annoying interruptions are :-).) In order to argue your 
point I think you're intentionally trying to misunderstand Bill. 
He didn't talk about getting a new PDU while sending his PDU. 
He talked about getting an incoming non-empty PDU while 
sending his data _buffer_. This buffer could be much 
larger than a PDU. You cannot deny that this may happen
with the scheme that you're arguing for. It cannot happen with the
scheme that we're proposing, since only empty PDUs would be coming
back while we're sending our (possibly much larger than PDUs) data
buffers.

> Once you send the PDU, there may still be some data left in the output
> buffer but it doesn't seem complex to switch at that point to waiting 
> to
> receive and parse the next received PDU. 

If you have noticed that a PDU is full, it is certainly possible to
switch to a receiving mode. Yet it is even simpler not to have to notice.

> One had better be able to switch
> from one process to another anyway as most implementations will handle 
> multiple connections which can be in diffent phases. 

I don't see the relevance of this argument.

> If you are willing
> to buffer all your output into a buffer and wait to send it until the 
> other
> side has stopped sending new parameters, then all you need to do to 
> change
> that to an implementation that sends what it has in the output buffer 
> as 
> it goes along is to have pointer into the buffer that indicates the 
> next
> byte to be sent. <PAT>

Not true, because as I'll start sending it, the other side may
send me new offers, thus preempting me from making similar offers
myself and instead requiring _responses_.

> The bit about sending empty PDUs is an effort to avoid that. We are 
> either
> in a sending-negotiation-text mode, or a parsing mode. We don't have to
> worry about strings being partially there, since we don't process until
> they are. We also don't have to worry about if our key/value pairs
> cross a PDU. The parser/negotiatior is MUCH simpler; it doesn't have to
> deal with all of that.
> 
> Letting the target answer while the initiator is trying to send an
> extended PDU adds the following complications (AFAICT):
> 
> 1) initiator has to make sure a key and the '=' don't get split on a 
> PDU
> boundry.
> 
> <PAT> One doesn't have to do that anyway. We can chose the rule to not
> offering a new (non-declarative key) when one has received a partial 
> key
> which seemed to be acceptable to a number of people. <PAT>

First I haven't yet seen anybody but you and myself speak about the 
declarative/non-declarative variation of this theme yet, so I wouldn't
call it "acceptable to a number of people" yet, at least not if
you include the parenthesized part. Overall, however,
a truly aware Sender most likely will check whether a key and =
got split or not. Because if it was split, and the other side
originates a new key, it is a violation. It would be nice to catch it.
And it definitely must check whether key=value fit fully or not.

> 2) if an initiator fills up a PDU, it has to stop processing & wait for
> the response. It also has to remember where it is so that it can finish
> sending that key/value next time.
> 
> <PAT> It has to remember where it was and finish the key/value next 
> time
> anyway as it can't send again until it gets a response. <PAT>

The difference is that in the scheme we're proposing we'd leave the
unsent key=value pairs sitting in the "leftover" buffer, and happily mark 
all those keys individually as "sent". So we'll consider them
processed, even though they may not have been put on the wire yet.
With your scheme, you may not mark them as sent, so there is also 
little point having them stay in the buffer. The only thing that 
can safely stay in the buffer is the remainder of the last key=value pair, 
if it was sent partially.

> 3) The target has to detect a split key/value pair on in-bound, and a)
> store the received text, and b) remember not to offer said key or any
> others.
> 
> <PAT> detecting an incomplete key is reasonably easy. And the target 
> can
> chose to do the simplist thing which is not to offer new keys when a 
> partial key is outstanding. <PAT>

Yes, easy. Still harder than never dealing with an incomplete key at all.

> 4) The target ALSO has to make sure ITS response doesn't fill up a PDU. 
> If
> it does, it has to remember where IT is and that it has more to send.
> 
> <PAT> In your approach the target can have a buffer with more than 
> one PDU worth of response in it. Once it starts sending from that 
> buffer
> it has to remember where it is and that it has more to send. This is
> not any different. There is a buffer with pointers into for end and
> last byte sent (or next byte to send). <PAT>

Yes, it is different, because keys can be marked "sent" and happily
wait their physical send-out in the buffer. With your scheme you
can't put them in a buffer and forget, because incoming data may
force you to go back and start mangling the values you had assigned
them...

> That's a lot of complexity and saved state. At least in my mind.
> 
> <PAT> I don't see much complexity difference here.

There is quite some difference in complexity, here. I think you're
trying not to see it.

<snip>

> <PAT> I agree. This means that plug fests probably haven't hit these 
> cases. <PAT>

That's why there is the famous Bill's experiment---set the PDU size
to 64 bytes and try negotiating.

> Does the statement of the perceived problem make sense, even if the
> opinion of the WG is to not use the empty PDUs? I would feel much
> comfortable with deciding to not use empty PDUs if folks said they
> understand the above point and they choose to address the problems
> differently.
> 
> <PAT> I want to point out that there are times in the protocol when 
> one will have to send an empty PDU. For instance, when getting a
> security key that is longer than a the Max PDU size (or even less
> than that - the sender isn't required to send Max size PDUs) or when 
> a device has no key-value pairs to send but its partner hasn't set
> the F bit. The issue is whether to require that there be a process
> for handing the right to originate keys (or in your proposal to send
> keys at all) keys from initiator to target and back which adds 
> its own complexity. I think the complexity will be
> of the same order as the existing situation. <PAT>

I think in both cases we do have a process to hand the right to
originate/send to the other side and back and forth again. We are
not trying to take this process away. The question is should the
"handing over of the right (to originate/send)" happen after
each PDU sent or should it happen after a data _buffer_ (possibly
containing more key=value pairs than fit in a PDU) sent?

Again a simple comparison to real life... Take the first text
in this message written by me. How would you have liked to delete
everything beneath it and to send the rest of your comments
in a new message? Again, include only as little as there is
until my next comment. That's when I'd interrupt you again
and potentially mess with what you were planning to say. 
Pretty frustrating, wouldn't it be? OK, to be fair, I couldn't 
do it as often as I could want to, since you'd be entitled to one 
PDU worth of uninterrupted text in iSCSI. But it isn't as clean
as "letting the speaker finish" (not the lecture, just the
"complete thought").

Martins Krikis, Intel Corp.

Disclaimer: these opinions are mine and may not be those of my employer.