RE: multiple connections

[Joshua Tseng <jtseng@NishanSystems.com>]

>One downside I see to this proposal is that for a host with multiple iSCSI
>nics, it will have to issue two commands (in this order): one to the "data
path
>nic" to describe where the data is to be read from/written to, and one to
the
>"control path nic" to send the command.  This would be communication to two
>cards instead of one.  Also, note that the communication to the data path
nic
>MUST complete before the control path nic sends the command, requiring some
>kind of sync mechanism.

I don't understand why you think two commands are necessary.  A single
command
down the command/control connection will work.  This is my view of how an
async model would work:

Data and command/control connections would be pre-established.  In the
model with one command/control and multiple data connections, the
command PDU could go first down the control connection, and have a
"data connection handle" field which would specify which data connection
will be used.  In a write command, unsolicited data may be part of the
command PDU, which would be sent down the control connection.  The target
would respond with a RTT using the data connection specified by the original
command, and all subsequent data PDU's would be sent down the specified 
data connection.  In a read command, the target would respond by sending
back data on the specified data connection.

>For the case of the command connection failure and fail-over to a new
>connection, I don't see how you can get away from the command counters.
When a
>fail over occurs, you will need some way of finding out what commands made
it
>to the target and which didn't.  The easiest way to do this is with command
>numbering.

Since commands must be delivered sequentially by iSCSI, the lost command is
the last command for which there was no reply or acknowledgement.  I think
the issue is how to detect a lost command, and I'm not sure if command
counters will help here either.  To detect a lost command, you either need a
timeout, or implement something similar to FCP's REC link service command.
Neither of these options is a flow control mechanism, and therefore does not
solve a major cause of the lost commands--the initiator overrunning the
target
with too many commands.  Another possibility is the command windowing
mechanism in the existing iSCSI specification.  But then how wide shall the 
target open the window to allow the initiator to send multiple commands?
And
at what threshold shall the difference between StatRN and ExpStatRN before
the initiator determines that the command has been lost?

An alternative proposal is for the target to communicate to the initiator
the
amount of buffer he has available at iSCSI login.  This is a static
parameter
dependent on the amount of memory available in the target.  It is up to the 
initiator to decide how many commands to put in flight simultaneously.  I
believe
the initiator (not the target, as currently specified by the existing iSCSI
spec)
is in the best position to decide how many commands to put in flight,
because
the initiator knows what commands it has coming, and more importantly the
size
of the command PDU's, including any unsolicited write data, which will
affect
the buffer usage at the target.  Additionally, the initiator knows what
recovery
mechanisms (if any) it has at its disposal in the event of a lost command,
and
can decide whether to be aggressive or conservative based on its specific
implementation.

Comments would be appreciated on this proposed model.

Josh

-----Original Message-----
From: Matt Wakeley [mailto:matt_wakeley@agilent.com]
Sent: Wednesday, September 06, 2000 1:17 AM
To: ips@ece.cmu.edu
Subject: Re: multiple connections


julian_satran@il.ibm.com wrote:

> Dear colleagues,
>
> With all the heated debate about multiple vs. single connection a request
I
> made a while ago got no significant reply (neither for nor against).
>
> The request was to consider a proposal made by Kalman Meth to reconsider
> the asymmetric model with the addition of a path selection made by the
> initiator.

One downside I see to this proposal is that for a host with multiple iSCSI
nics, it will have to issue two commands (in this order): one to the "data
path
nic" to describe where the data is to be read from/written to, and one to
the
"control path nic" to send the command.  This would be communication to two
cards instead of one.  Also, note that the communication to the data path
nic
MUST complete before the control path nic sends the command, requiring some
kind of sync mechanism.


> This proposal allows removing the command counters - as commands use a
> single TCP connection. The single connection can also be a shared
> data+control connection.

For the case of the command connection failure and fail-over to a new
connection, I don't see how you can get away from the command counters.
When a
fail over occurs, you will need some way of finding out what commands made
it
to the target and which didn't.  The easiest way to do this is with command
numbering.

>
>
> In case of multiple connection the data path to be used is selected and
> maintained until the command ends.
>
> Thanks,
> Julo

-Matt Wakeley
Agilent Technologies