RE: iSCSI: "Wedge" drivers

[julian_satran@il.ibm.com]

Earnie,

Thanks - I see your point. There are a multitude of controllers that
virtualize a set
of controllers into a "super-unit" and could have the same issues.
But the wedge driver is in this case left only with the clustering issues
and
will not have to do anything with transport?

And thanks for your thoughts on the asymmetric/symmetric issue.

Julo

"Pistor, Ernie" <ernie.pistor@storageapps.com> on 29/08/2000 20:33:24

Please respond to "Pistor, Ernie" <ernie.pistor@storageapps.com>

To:   ips@ece.cmu.edu
cc:    (bcc: Julian Satran/Haifa/IBM)
Subject:  RE: iSCSI: "Wedge" drivers




Julo (and others)-

I believe David's points 1 and 4 are directly related; the Clariion array
(and others that I know of) is active-active in that both controllers will
accept SCSI commands, but active-passive in that each LU is explicitly
owned
by one contrller or the other.  Commands for a LU to the non-owning
controller are rejected, and AFAIK a controller has no knowledge of
commands
that are active (through the other controller) to LUs it does not own.
Ownership of a LU can be changed to the other controller, but the method is
vendor-specific and requires a wedge driver, as it must be initiated by the
host system.  Note also that it may be desirable to change LU ownership for
reasons other than controller failure (i.e. host-side HBA failure, where
you
want to move a LU to the other controller so you can still access its
data),
so that, even if the array did support internal failover (I think Clariion
does as a not-so-widely-used option) a wedge driver would be desirable.
Enough vendor-specific babble... the intent was to help clarify David's
points through a bit of elaboration.

As for multiple connections, I cast my vote for the asymmetric approach as
long as it still permits us to support one connection per session (as in
Kalman Meth's proposal).  As you pointed out in an earlier message, a
single
control connection inherently takes care of command ordering without the
reference number; command processing is simplified.  As far as (control)
connection recovery is concerned, an alternate approach (to the one
specified in Kalman Meth's proposal) that does not require that a new
connection be opened immediately might be to designate the lowest-numbered
non-failed connection as the control connection.  There would need to be
some sort of message passed to inform the other end of the new control
connection by the side that first detects the broken connection, followed
by
the rest of Kalman's recovery scheme (Query/Sync) or some variant.  The
advantage to this approach is that, in the multiple connections case, it
could prevent the target from cleaning up its session state upon control
connection breakage; I don't believe Kalman's proposal specified the
target's actions if it discovers a dead control connection, and in the
spirit of the 00 draft my assumption would be cleanup (otherwise the target
is left with a "zombie" session if the initiator does not log in again).
This isn't an explicit proposal, but it's something that might be worth
thinking about...

Ernie Pistor

-----Original Message-----
From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com]
Sent: Tuesday, August 29, 2000 10:54 AM
To: ips@ece.cmu.edu
Subject: RE: iSCSI: "Wedge" drivers




David,

I understand 3 and I somehow expected it. If I would have to reimplement
this
under iSCSI I would choose a plug-in in iSCSI to do it (a policy module)
and
iSCSI would clearly support such a construct (and make it also
future-proof).

I have some trouble with 1 and 2.  In whatever design you choose at the
array
level you have to have some sharing as some commands are targeted to a LU
and all it's queues (i.e. separation per initiator is never complete if you
take SAM seriously).
However some design might choose to share only for those commands - and
talking to you
with you chair hat off - I don't think that you are with one of those (:-

I am no sure I understand 4 either - if the array does no "sense" the fail
over.

Now talking to you with your hat on - we are ready to get to the drafting -
but I feel
that we are still left with the symmetric/asymmetric dilemma and I would
appreciate
input:

The asymmetric solution is simpler for those using a single connection.

Is there any serious drawback?

Dear Colleagues - Please give us feedback.

We can choose only one solution?  Which is the right one?

Julo

Black_David@emc.com on 29/08/2000 16:26:49

Please respond to Black_David@emc.com

To:   Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu
cc:
Subject:  RE: iSCSI: "Wedge" drivers




> While many of you keep telling me that wedge drivers won't go away nobody
> gave one good example to support that assertion.

With my co-chair hat off, here are four, two from the mail exchange:
(1) Sharing SCSI state across array interface processors involves some
     work.  Use of multiple SCSI connections and wedge drivers
     for fault tolerance and load balancing will persist in systems that
     choose not to follow that implementation path.
(2) If iSCSI is implemented in hardware, so that a system has multiple
iSCSI
     HBAs rather than multiple NICs with a common iSCSI software
     stack, multiple SCSI connections and wedge drivers are again
     the path of least resistance to fault tolerance and load balancing.
And two new ones:
(3) Naive greedy load balancing is not optimal.  Significant improvements
     are possible based on an understanding of how the storage behaves.
     PowerPath contains Symmetrix-specific logic that does this,
     and I don't think that's appropriate for standardization.
(4) Some arrays need additional support for fail-over.  Clariion uses
     an active fail-over architecture in which the array must be
     instructed to fail-over, and the fail-over occurs across separate
     SCSI connections.  I don't think that the Clariion-specific
     logic that does this is appropriate for standardization.
As I wrote earlier, I have no problem with specifying a session abstraction
for iSCSI, but it won't eliminate wedge drivers.

With my co-chair hat on, the current consensus is to specify a session
abstraction and review the result.

--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 435-1000 x75140, FAX: +1 (508) 497-6909
black_david@emc.com  Cellular: +1 (978) 394-7754
---------------------------------------------------