Re: Avoiding deadlock in iSCSI

[Pierre Labat <pierre_labat@hp.com>]

julian_satran@il.ibm.com wrote:

> Pierre,
>
> That is the essence of Kalman's proposal (and that differentiates it from
> draft-00 and from an unplanned-allegiance that we
> discussed in the loby of a hote in Adelaide 100 years ago.

I disagree, there is a fundamental difference with the Kalman draft:  in the
proposal below
always cmd/data/status goes on the same TCP connection. It is not the case
in the Kalman
draft. The fact that in the Kalman draft the end of data notification and
command status
comes on two different TCP connection (thus 2 NICs most of the time) implies
an important
CPU overhead on the initiator to manage the READ command completion as
described
by Somesh Gupta. You need for each READ to work accross two NICs. You will
double
the work on each read completion. Moreover you will double the number of
cache misses
because you need to read completion information from two NICs. And cache
misses
are really stalling your CPUs.
It's a reason for what i can't agree on the Kalman proposal. You don't have
this penalty
in the proposal below.

Regards,

Pierre

>
>
> Julo
>
> Pierre Labat <pierre_labat@hp.com> on 13/09/2000 11:48:58
>
> Please respond to Pierre Labat <pierre_labat@hp.com>
>
> To:   ips@ece.cmu.edu
> cc:    (bcc: Julian Satran/Haifa/IBM)
> Subject:  RE: Avoiding deadlock in iSCSI
>
> >From the discussion going on the symmetric versus asymmetric model and
> dead lock avoidance, here is  a proposal that tries to combine (as far
> as possible...)
> the advantages of both models.
>
> The proposal is a slightly modified symmetric model where
> the symmetric model incorporate an idea of the current
> asymmetric model.
>
> This idea is that the initiator indicates to the target
> on which connection the next command in order arrives.
> The asymmetric model does that using an extra connection
> transporting the commands, and for each command in the header
> there is the connection number where the data will come.
>
> Description of the proposal:
> ---------------------
> In the command header a field is added. This field
> "next connection" indicates the TCP connection (connection id)
> in the session on which the next command will arrive.
> This field builds a linked list of commands across the several
> symmetric connections of the session.
>
> The first time the initiator posts a command, it determines
> the connection on which it post the command and the connection
> on which it will post the next command.
> Then for each command the initiator determines only
> the connection it will use to post the next command.
> For each command, the connexion id of the next command is added in the
> header
> field "next connection" of each command.
>
> Doing that, leads to the same algorithm as in the asymmetric model
> to process the commands on the target side:
> - a thread of execution pulls the cmd/data in order from the TCP
>   windows. After pulling a cmd/data, it looks at the "next connection"
>   field and move to the next connection to pull a new cmd/data.
>   There is no need of extra storage out of the TCP windows
>   and no need to re-order. In the asymmetric model the thread
>   empties the command connection, here it follows the chain of commands.
>
> The advantage of this proposal are:
> - on the target, as opposed to the original symmetric model, it avoids
>   the re-ordering process of the commands once extracted
>   off the TCP window.
> - it keeps cmd/data/status on a same connexion avoiding
>   the synchronization penalty for the READS on the initiator (described
>   by Somesh Gupta)
> - it saves one TCP connection compared to the asymmetric model.
>
> Anyway any ordering implementation adds overhead, thus, as in most
> cases (disks) it is not required, It would be possible to deactivate it.
>
> Regards,
>
> Pierre