Re: ISCSI: Urgent pointer consensus

[Daniel Smith <dfsmith@almaden.ibm.com>]

Matt Wakeley wrote:
> 
> > send CDB PDU A (write)
> > send CDB PDU B (read)
> > rx r2t for A              <----may get lost
> > send CDB PDU C (read)
> > rx data for B
> > send data for A
> > rx data for C

Thanks for explaining your thinking.

This is a very compelling scenario where the urgent pointer would improve
performance (or at least, lower on-NIC/kernel-skbuf memory use).

Personally, I'd be wary of skipping a PDU that I didn't know about.  I can
think of dubious and unlikely scenarios involving missing PDUs containing
AENs or iSCSI messages saying things like "data encryption key renegotiated",
"added 27 to all LUNs" and "read B had corrupted sectors that I think I've
corrected properly".  Pretty far-fetched, but not totally impossible.

I've been doing a bit of reading, and have come to agreement that should we
wish to mark message boundaries, the urgent offset is probably the correct
place to do it.  (Comer notes that the occurrence of URG is "rare", but both
Comer and Wright/Stevens concur about its use being totally application
driven.)

The urgent offset doesn't match this application in two minor ways though. 

The first is that the offset is supposed to indicate the most recent data
that have been declared urgent.  This is a stack-centric view, not a
network-centric view.  So if the urgent pointer is continually more than
(64K-n bytes) away (n smallish) then the actual pointer value will never
propagate out of current stacks.  (This is very likely in iSCSI with a window
>64KB.)

The second concern I found in one of the exercises in Stevens (Question
26.2, "TCP/IP Illustrated, Vol 1") which concerns the treatment of urgent
mode.  (Urgent mode is the state that TCP goes in when the urgent offset is
present.)  When in urgent mode, the TCP stack is obliged to ACK every
segment from the sender, even ACKs!  This is fine when only one side is in
urgent mode.  In iSCSI, both sides would be in urgent mode.  Does anyone
know how this would be handled in real life?  I'm sure they must have worked
out that an ACK storm is a bad thing, but URG might be so rare that it was
never extensively analysed.

I think in the end, the actual use of URG is a personal choice.  My choice
is optional.

Daniel Smith.

P.S., the urgent offset is carried in the regular TCP header, and does not
increase the number of bytes sent (except for the extra ACK segments).  My
oversight in the previous email, sorry.
-- 
IBM Almaden Research Center, 650 Harry Road, San Jose, CA 95120-6099, USA
K65B/C2 Phone: +1(408)927-2072 Fax: +1(408)927-3010