RE: multiple TCP connections

[Stephen Byan <Stephen.Byan@quantum.com>]

Thanks to all who answered my "help me figure out what I don't know"
questions regarding multiple TCP connections.

To summarize my current understanding:

To my surprise, neither TCP, IP, nor Ethernet have mechanisms for striping
traffic across multiple interfaces when that traffic is part of a single TCP
connection. (There may be an exception for ethernet in the letter of
802.3ad, but in terms of practical implementations available in the near
future, there's no striping avaiable for a single TCP connection.) 

However, there are mechanisms for striping traffic from multiple TCP
connections.

We are given the constraint that we can't change TCP.

If we assume a requirement to stripe traffic between a single muli-homed
host and a single LUN on a multi-homed device across multiple paths, then we
need to turn that traffic into traffic over multiple TCP connections.

One way of doing this would be to use one TCP connection per SCSI task.
(Think HTTP.) To minimize the costly overhead of this approach, we'd
probably have to consider something like Transactional TCP. Moveover, this
would not increase the transfer speed for a single command. I don't find
this approach very palatable.

Another way of distributing the traffic over multiple TCP connections is to
create the notion of a session, and bind multiple TCP connections to that
session. Architecturally this looks yucky, but I prefer sessions over the
alternative of one connection per command.


So I agree that if we accept the requirement as stated above, then sessions
are the logical answer.

<end of summary>


Discussion of requirements:

I happen to agree with the requirement to stripe the traffic between a
single muli-homed host and a single LUN on a multi-homed device across
multiple paths. For a small, cheap iSCSI implementations near the device
level, I'd like to stick with 1 Gb ethernet even as 10 Gb becomes available,
in order to minimize costs. Customers really like multiple paths to their
devices, so device-level implementations will require two interfaces for
this reason alone. Finally, note that media data rates are always
increasing, and we're not that many years away from hitting 1 Gb/s. 

We can argue whether the storage interconnect really needs to match the
media rate, since once the drive starts seeking the average data rate drops
by orders of magnitude. Regardless of the technical merits, I think
interface speed sells in the storage market, so I think customers will
continue to demand interface speeds that match the media rate. Consequently
I think that low-end iSCSI devices will have a requirement for interface
speeds in excess of 2 Gb/s within the next few years, and would best meet
this requirement by striping across two 1 Gb ethernet interfaces in an iSCSI
session.


Complaints about current definition of an iSCSI session:

I have one concern about the current definition of an iSCSI session. I'd
prefer a striping mechanism that would support striping of the data transfer
for a single command across multiple interfaces. As the spec stands now, all
messages for a single command have allegiance to one TCP connection, and
therefor the all the data transfer for that command must occur over one TCP
connection. 

Given the current architecture of iSCSI, I see the need for this allegiance,
since IMHO hardware DMA acceleration requires access to the state of the
iSCSI command. However, if we adopt VI/TCP as the enabler for hardware DMA
acceleration, the data transfer state is (almost) self-contained in the VI
segment header, and I think it becomes feasible to remove the requirement
that the data transfer maintain allegiance to the TCP connection that
originated the SCSI command. Then we could stripe the data DMA traffic
across multiple interfaces, while still keeping all the control and status
traffic on the originating TCP connection.

Regards,
-Steve

Steve Byan
<stephen.byan@quantum.com>
Design Engineer
MS 1-3/E23
333 South Street
Shrewsbury, MA 01545
(508)770-3414
fax: (508)770-2604