RE: iSCSI: Markers

["WENDT,JIM (HP-Roseville,ex1)" <jim_wendt@hp.com>]

> From: Jonathan Stone [mailto:jonathan@dsg.stanford.edu]
> ...
> Even if a COWS data-touching loop is folded into another data-reading
> loop, COWS encoding/decoding still dirties all the cache lines being
> encoded or decoded, and will therefore has the same overhead as an
> additional copy.

Since inline COWS encoding only has to write back chain links into buffer
locations that contain the flag value (CFP), isn't the dirty-cache-line
overhead proportional to the frequency of occurrence of the flag in the
buffer? Given a 32-bit flag, it would seem that the probability of
encountering a flag in the buffer would normally be quite low, and few if
any cache lines would be dirty, hence the overhead would be just in the
reads from the buffer - which could be hidden in another data reading loop.

COWS does require writing an initial flag and chain link just in front of
the buffer, but either a) that cache line may already be dirty because other
PDU header fields have just been written, or b) part of building the PDU
header can include writing the flag value and a default link chain that
spans the whole PDU.

[A follow up discussion point is that the original COBS scheme also appended
a flag value to the end of the PDU, so that the initial chain link (CLCE) or
final link chain could point to something associated with the current
buffer.

> Performing COBS encoding/decoding in an outbouard accelerator also
> makes it impossible for those of us who don't trust our I/O busses to
> perform a `trust, but verify' integrity check: e.g., recomputing the
> TCP/IP checksum over the data stream as the HBA-cum-NIC deposits it
> in host memory.

Are you assuming that the HBA/NIC is depositing TCP segments into host
memory, as an iSCSI HBA/NIC would be transferring iSCSI PDUs.
What is your preferred model for integrity checking of NIC/HBA transfers
across I/O buses in the face of some offloading of transport or ULP
functionality?

> Indeed, the original COBS implementation(s) provided only framing;
> COBS framing was wrapped outside an end-to-end integrity check.

This is also true for iSCSI/COWS, since the flag separates iSCSI PDUs and
each iSCSI PDU contains (optional) header and data digests (CRCs).

> I beleive Stuart is still on vacation, but I can ask him to comment on
> these issues when he gets back, if that would be worthwhile.

Absolutely.

Jim