RE: iSCSI: Partial Session Consensus

[somesh_gupta@hp.com]

David,

I still have to dig through all the messages (my mail system
was broken for a day), so I don't know if there is anything
that will change my mind.

But let me record a preliminary objection to the (1). I think
there are benefits to command reference numbers and sliding
window -

First, I would like to argue assuming that the current
implementations of targets and initiators may not necessarily
be the best way of achieving max throughput on iSCSI,
considering the range of latencies that are possible.
Then of course, memory has become cheaper - and protocol
hardware implementations more and more complex.

So what may be ideal is where we allow the initiator and
target to be streaming commands, data and responses, with
some means of avoiding trouble all the time.

1. A management of the command queue depth should enable
better utilization of the pipeline by enabling streaming
of commands without going into the recovery case all the
time. 

   iSCSI is better for this that the SCSI layer. The SCSI
   layer can keep posting commands and data buffers to the
   iSCSI adapter, and the iSCSI adapter can then manage
   when to send more commands. Otherwise, this information
   would have to bump up to the host, and the host would
   then post more commands to the iSCSI adapter - increasing
   workload and latency

   It also lets performance vary by amount of memory in the
   array etc rather than depend on some historic algorithm
   implemented in the initiator.

2. Numbering in general will make recover from command drops
   easier. As mentioned by Costa, myself and others, if the
   initiator drops a command, it drops all commands from
   then on, sends a notification to the initiator, who
   then sends an ack of the condition, and starts sending
   commands from the dropped command. The target may
   accompany the drop notification with a reduction in the
   command window.

This minimizes the possibility of command queue overflow
(and deadlock), while allowing as much throughput as
possible based on implementation (algorithm and memory)
choices.

Somesh

> -----Original Message-----
> From: Black_David@emc.com [mailto:Black_David@emc.com]
> Sent: Tuesday, September 12, 2000 9:24 AM
> To: ips@ece.cmu.edu
> Subject: iSCSI: Partial Session Consensus
> 
> 
> In reading the mailing list traffic on Asymmetric
> vs. Symmetric models for multiple TCP connections
> in an iSCSI session, I don't believe that there is
> consensus on the issue phrased in that fashion.
> 
> I believe that there is consensus on a couple of
> underlying issues:
> 
> (1) An iSCSI session containing a single TCP connection
> 	should not be required to use the currently specified
> 	iSCSI command reference numbers and sliding window
> 	mechanism because TCP will deliver commands in order.
> (2) Use of more than one TCP connection per iSCSI session
> 	is OPTIONAL.
> 
> The consensus on (1) is rough; if anyone disagrees with
> this for a reason other than wanting to use the Symmetric
> model when an iSCSI session contains multiple TCP connections,
> please say so on the list.  My reading of consensus is
> based on the fact that much of the recent discussion of the
> Asymmetric model has been motivated by single connection
> sessions, whereas essentially all of the recent discussion
> of the Symmetric model seems to have been focussed on
> multiple connection sessions.
> 
> The consensus on (2) is sufficiently long standing to be a
> closed issue.  The resolution to the "deadlock" scenarios posted
> by Costa is that the target must not stop reading from the TCP
> connection - SCSI provides means for a target to throw away
> things it can't deal with (e.g., TASK SET FULL), and with
> regard to the ordered command issues, I believe Steve Byan
> is correct when he says:
> 
> 	I think this is a T10 bug, and iSCSI should defer to 
> T10 to fix it.
> 
> In addition to Steve noting that disks tend not to use ordered
> commands, I would note that a single tape device/drive tends
> not to have multiple simultaneous initiators sending commands
> to it, both of which limit the practical impact of this potential
> problem.
> 
> I don't see consensus on the model for iSCSI sessions that
> contain multiple TCP connections.  The consensus on (1)
> above for single TCP connection sessions does not take the
> Symmetric model out of consideration for multiple TCP connection
> sessions.  If multiple sessions are negotiated on connection
> establishment, command reference numbers could be added to
> subsequent headers as a result of that successful negotiation.
> In order to make progress, this additional complexity should not
> be used as an argument against the consensus in (1) for single
> connection sessions.
> 
> I hope the consensus on (1), or something close to it holds,
> as if it does not, we may have to form an offline design team or
> teams to work on this set of session issues, and that could take
> some time ... meanwhile, discussion of Asymmetric vs.
> Symmetric for multiple connection sessions should continue.
> 
> --David
> 
> ---------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 42 South St., Hopkinton, MA  01748
> +1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
> black_david@emc.com       Mobile: +1 (978) 394-7754
> ---------------------------------------------------
>