RE: iSCSI: "Wedge" drivers

["John Hufferd/San Jose/IBM" <hufferd@us.ibm.com>]

David,
again we mostly agree.
I do not know if the iSCSI initiator finding out about the other "equal"
targets is awkward or not (I expect this is in eye of beholder) however,
having explicit exchange of information perhaps at Login time, seem to be
quite acceptable.

What I feel we are talking about is NOT whether ALL wedge drivers will go
away -- they probably WILL NOT, because of various value added functions we
all think is important to have in our controllers -- but instead we are
talking about whether we think there is a layering type approach, which can
perform some generic load balancing across NICs and perhaps some generic
alternate path retry, and where that layering is within the iSCSI driver.

One contrary argument to the above, is that since we need Vendor Specific
Wedge Drivers anyway, we might as well use them for load balancing and
alternate path recovery.

Let me argue against that for a moment.  As standards occur in this
industry, there has been a consistent march to make what was once a vendor
specific feature into a generic feature to all/most vendors.  So I believe
that it is right within the normal standards march for there to be constant
changes to the Wedge Drivers as new things are brought in, and old things
become standards and located into some "normalized" point in the common
structure.  With this is mind, it seems to me, that we can move to a
position (perhaps over time) where the iSCSI Device Driver is able to
perform the load balancing, and alternate path recovery, yet leave other
functions for a higher level Wedge Driver to perform.  This tends to keep
the upper layers, simpler and easier to write and maintain.

If the above concept seems reasonable, then we need to keep the Multiple
Conversations per Session.

Again let me state my belief that the implementations that will ship out,
originally, will be Single Conversations per Session, especially since it
is easier and the default.  This will mean that some of the current Wedge
Device Drivers will remain unchanged, and therefore, alternate path
recovery with load balancing will be done, (or not) at this layer.
However, over time, I think this will change -- as the various vendors have
time to implement the Multiple Conversations per Session.  This will
permit, in some cases, the elimination of Wedge Device Drivers, but
probably just the simplification of the Wedge Device Drivers.


.
.
.
John L. Hufferd


Black_David@emc.com@ece.cmu.edu on 08/28/2000 09:24:06 AM

Sent by:  owner-ips@ece.cmu.edu


To:   ips@ece.cmu.edu
cc:
Subject:  RE: iSCSI: "Wedge" drivers



Still with my co-chair hat off, let me try to summarize,
rather than quoting and responding:

If one assumes that an iSCSI session spans multiple
interface processors on the storage side, then those
interface processors have to share SCSI state (e.g.,
ranges of data for each I/O on which Ready to Transfer
has been vs. needs to be issued) in some fashion so
that the other processor has it when failure causes
it to become involved.  Use of a load balancing switch
does not change this situation.

Julian (IIRC) has observed that the corresponding state
sharing is done for mainframe storage (e.g., I/O completion
can come back on a different link than the one used for
initiation), but it is not done by any SCSI implementation
I'm aware of.  If nothing else, that's proof by example
that any box (EMC, IBM, etc.) that supports mainframe
storage is in principle capable of spreading a SCSI
connection across more than one interface processor.
This is additional implementation complexity and effort,
as state sharing across failure domains is not easy to
get right, and this is a new form of sharing that SCSI
implementers have little experience with.

A similar issue occurs on the host side.  If the system
is equipped with NICs so that iSCSI above TCP/IP is entirely
in software, then spreading a SCSI connection across multiple
NICs should work fine.  OTOH, if the system has HBAs that
implement much of iSCSI in hardware in addition to TCP/IP,
then not sharing iSCSI state across HBAs is again easier
to build (and entails a wedge driver).

I'm trying not to take a position for or against iSCSI
sessions (e.g., I can see a strong rationale for using
them with tape, where wedge drivers don't work and the
fault tolerance requirements aren't as stringent).  The
point I wanted to make is that engineering and implementation
concerns make it highly unlikely that iSCSI sessions would
remove the need for wedge drivers for disk.  My goal
is to contribute to a clear understanding of the technical
rationale behind sessions, for the requirements document
if nothing else.

Several minor issues ...

John described a model in which an initiator discovers
that multiple TCP/IP connections are part of the same
session.  That seems awkward by comparison to having
each participant tell the other what the alternate
transport addresses are (or how to find them) as the
latter avoids having to splice a new connection into
one that's already actively doing I/O.  SCTP sets up
multiple addresses in this fashion, FWIW.

EMC error retry does not always require an EMC-written wedge
driver.  For example, Symmetrix works with the Veritas DMP
wedge driver.

There's no free lunch on configuration -- since wedge
drivers are not going to go away, iSCSI sessions become
something configured "in addition to" wedge driver
configuration rather than "instead of".

--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 435-1000 x75140, FAX: +1 (508) 497-6909
black_david@emc.com  Cellular: +1 (978) 394-7754
---------------------------------------------------