ISCSI: asymmetric vs symmetric and multiple connections

["Matt Wakeley" <matt_wakeley@agilent.com>]

David thinks we have come to "consensus". I'm not so sure. I would like to
summarize what the issues are with symmetric vs asymmetric and multiple tcp
connections.

definitions:  Symmetric model means commands and their associated data flow
down any of the TCP connections that make up a session.  Asymmetric model
means that all commands flow on one TCP connection, and all data flows on the
remaining TCP connections that make up the session.

Note that with the above definitions, an iSCSI session with a single tcp
connection is by definition a symmetric model.

Asymmetric model pros:

  - does not require a "command reordering" mechanism.
  - commands (relatively small messages) are generally not flow controlled
    by TCP like they would be if multiplexed with large data transfers.
  - may be easier to implement in hardware - no multiplexing of commands
    and data on the same TCP connection.

Asymmetric model cons:

  - requires a minimum of two TCP connections: one for commands, one for data.

  - iSCSI sessions using multiple TCP connections across different iSCSI NICs
    to provide aggregation of links requires double the control and setup on
    the processor I/O bus to perform a SCSI I/O.  This is because one NIC must

    be set up to send the command, the other for the data.  This would not be
    and issue for an iSCSI NIC that had multiple MACs or connections to the
    fabric.
  - may make the implementation of "immediate data" on targets difficult
    (the data could arrive before the command).
  - may require more TCP context storage due to a minimum of two TCP
    connections per session (but not nearly as much as those advocating
    a connection per LU!).

Symmetric model pros:

  - the "double control overhead" of the asymmetric model does not apply.
  - all TCP connections are the same - may be easier to implement in
    hardware because there are no "special" connections.
  - implementation of immediate data easier on targets - data always follows
    command on same TCP connection.

Symmetric model cons:

  - requires command reference numbers to re-order commands on multi
    TCP connection sessions (could be optional for a single TCP connection
    iSCSI session).
  - commands may be stalled behind data due to TCP flow control.
  - sliding window for commands implemented only to address the case
    where a TCP connection dies. (not to address the SCSI command queue
    overflow issue.)

-Matt Wakeley
Agilent Technologies