Re: iSCSI: Markers

[Stephen Bailey <steph@cs.uchicago.edu>]

> assuming iWARP is not an available option

What John is referring to as iWarp is actually `TUF' (TCP ULP
Framing).  iWarp is a putative RDMA on IP transports protocol, which
is not what we're talking about.

All the TCP framing proposals we're discussing have:

  1) an in-stream format

To this, TUF adds:

  2) a TCP sender segmentation behavior

The in-stream formats so far are:
  o fixed-interval markers (FIM)
  o random key/length headers (TUF's FPDU)
  o COWS header + marker knockout chain

In addition to Julian's COWS proposal, Costa & I also did one:

  http://www.cs.uchicago.edu/~steph/draft-bailey-tsvwg-cows-00.txt

It's substantially similar to Julian's, with a couple differences:

  o pointers are only forward
  o payload length is 16-bit length (trivial to make it up to 2^30-1 tho)
  o payload size is byte-granular
  o marker is defined a priori (I like Julian's idea of being able
    to exchange markers if desired---we'd add that)

Our COWS was defined as an alternative to (replacement of) TUF's
current in-stream format.  Compelling benefits of the COWS in-stream
format from our standpoint are:

  o the SAME in-stream format is works with or without TCP sender
    segmentation support 
  o no chance of a false positive from key/length validation

The first point is particularly important.  COWS can be used to
validate the PDU contaiment property if the TCP sender is
TUF-compliant, and it can be used for marker-like (better, actually)
resynchronization if the TCP sender is stock.  Before the COWS
proposal, we always had two different in-stream representations
(markers, & key/lengths) for different purposes.

The primary disadvantage of the COWS in-stream format is:

  o prohibits 0-touch software senders

I had suggested that if everybody liked COWS, we would update TUF to
use it, and even if TUF remains experimental, iSCSI could define the
same in-stream format for its framing (replacing markers).  That way,
there would really be only ONE in-stream format in the universe,
though it might be redundantly defined in two places.

For iSCSI, a single in-stream format that supports both modified and
stock sender approachs seems nice.  Implementations can easily migrate
from stock to modified TCP senders (ask one of the authors of the TUF
draft if you don't understand why modified TCP sender segmentation is
good).

For the rest of the world it would be nice (essential in my opinion)
if the same framing technique could be used for iSCSI and other
protocols (e.g. iWarp).  System vendors that support iSCSI just might
want to support NAS too, which could mean DAFS (on iWarp).  Chip
vendors CERTAINLY want to maximize return from their TOE investment
which means being able support various accelerated protocols with the
same hardware.  Wouldn't it be stupid if we had a proliferation of
framing alternatives just because the world originally seemed flat?

The tradeoff of 0-touch versus a single format clearly has powerful
arguments pulling either way.  I'm personally ambivalent.  Mostly, I
want to make sure everybody in the iSCSI community is sufficiently
informed about what's at stake.

Steph