RE: iFCP vs FCIP

["Venkat Rangan" <venkat@rhapsodynetworks.com>]

Josh,

It seems FC switching and networking technology is driven primarily by the
needs of back-end storage subsystems. If we look at design choices that FC
has adopted, it is very clear that they focus on smaller networks (you
typically don't expect thousands of storage subsystems connected into a
common shared SAN with cascade of switches, behind your hosts even in a
large data center), with very low media loss rates (frame corruption rates
of 10^-16 is typical), smaller latencies (10us to 100us). Given the
evolution from a bus architecture (SCSI), the number of targets discovered,
the in-order delivery of data frames, low latency across the network are
"assumed" by applications. Whereas it is natural to set up an asynchronous
request on a TCP socket, most applications assume low latency and issue a
synchronous request to a storage subsystem. Block-level access based
applications have not assumed the same modes of failures as a file-level
access based applications have.

An example is out-of-order delivery of exchanges. Creating an FC mesh with
FSPF (as described in FC-SW-2) can cause out-of-order delivery of FCP
exchanges. End points are supposed to deal with out-of-order delivery, but
very few SCSI drivers and target mode devices appear to deal with this. So,
fabric vendors are forced to guarantee in-order delivery; one way they do
this is to ensure that FSPF routes remain "fixed" and a route change can
only occur after an R_A_TOV time elapses without any I/Os on the ISL.

Another example is approach to flow control and congestion avoidance. Flow
control is addressed using buffer-to-buffer credits, but congestion
avoidance in ISLs is something not really addressed. The solution is to use
Class-4 service and some form of VC_RDY to avoid this. Overprovisioning ISLs
is something that is rarely done. Once again, block storage being at the
bottom most layer in the information hierarchy of an organization, requires
robustness and predictability in performance. FC has targeted these issues
and addressed them well. It is fairly easy to achieve upwards of 90% link
utilization on an FC-AL, but much harder to do in an Ethernet segment.

Extending storage over a wide area either with iFCP or FCIP or FC-BB-2 needs
consideration of how applications are to deal with these problems and
issues. Without block-level applications adapting to wide-area networking
(loss rates of 10^-4, latencies of 100ms), it becomes harder to take care of
these issues in transport layer. Although iFCP and FCIP are both workable
technologies, I'm not sure if a mere address translation, or setting up a
tunnel is adequate. Again, what guarantees can we provide about in-order
delivery at the FCP Portals of iFCP?  Maybe block-level access in host
drivers and kernel block-level I/O will take care of these over time. But
until then, FC products and fabrics will continue to have its role. In that
respect, FC over ATM (FC-BB-2) appears to be  closer to the original design
goals of FC, compared to iFCP. BTW, I assume that iFCP handles FC-AL
correctly, by implementing an FL_Port equivalent.

Venkat Rangan
Rhapsody Networks Inc.
http://www.rhapsodynetworks.com


-----Original Message-----
From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
Joshua Tseng
Sent: Wednesday, November 29, 2000 10:55 AM
To: Murali Rajagopal; Ips (E-mail)
Subject: RE: iFCP vs FCIP


Murali,

>
> With my TC hat off:
>
> Charles observation that FCIP's goal to maintain transparency
> within the
> switching FC Fabric is correct as far data transport is
> concerned. However,
> there is a clearly defined architecture defined in FC-SW-2
> standards that
> allow a device such as FCIP to connect to a border switch. In
> other words,
> from a routing standpoint the FC fabric is certainly aware of
> a hierarchial
> network and is supported jointly by the FSPF routing protocol and the
> FSPF-backbone routing protocols. This OSPF-based hierarchial
> model provides
> a lot of flexibility to the nature of the FC backbone networks. TCP/IP
> happens to be one of the many possabilities. (Other
> possabilities include FC
> directly over ATM and SONET as defined in the ANSI T11 FC-BB
> standards)

Is the possibility of tunneling FC over ATM and SONET deemed to be
an advantage for Fibre Channel networks?  TCP/IP has been networking
over these transports for many years.  It seems if I can map Fibre
Channel to IP (as iFCP does), then I can leverage all of the physical
network infrastructure that TCP/IP currently runs on.

>
> The second plus of this model is that it allows any type of
> traffic and
> allows for a very simple almost stateless (from FC
> point-of-view) behavior.
> This directly translates to scalability. The comment made by
> someone in this
> thread about FCIP being limited is inaccurate- it is in fact
> the opposite.

I fail to understand how this changes or improves the scalability
of FC. FC has a hard limit of 239 switches, period.  FCIP doesn't
change this.

>
> Finally, Joshua's comment on the small number of switches in
> a FC SAN is an
> observation from the past and this is rapidly changing as
> evidenced by the
> growing size of SANs in Data Centers.

I have worked on stable OSPF networks comprising of more than
800 routers and switches.  When interconnected with EGP routing
protocols, IP is virtually unlimited in scale.  I do not think
Fibre Channel and FSPF can ever approach this scalability.  There
are stability problems right now with FSPF implementations.  Perhaps
in time, they will be patched up, but why wait for FC networking
protocols to be debugged when there are many stable and tested OSPF
implementions out there?

Josh

>
> -Murali Rajagopal
> LightSand Communications
>
>
> -----Original Message-----
> From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
> Charles Monia
> Sent: Tuesday, November 28, 2000 7:19 PM
> To: Ips (E-mail)
> Cc: David Robinson (E-mail)
> Subject: RE: iFCP vs FCIP
>
>
> Hi Folks:
>
> The issue is that the design goals and underlying network models are
> fundamentally different. Essentially, FCIP's goal is to provide a
> transparent conduit between Fibre Channel fabrics while
> iFCP's goal is ULP
> transparency between N_PORTs.
>
> As a result, in iFCP, the fabric-wide services provided by FC fabric
> elements (and often implemented with proprietary protocols)
> are replaced by
> standard, IP-based equivalents. For that reason, an iFCP
> gateway does not
> need to recognize or provide facilities for servicing inter-switch FC
> protocols, such as those for zoning, naming and routing.
>
> Charles
> > -----Original Message-----
> > From: Joshua Tseng [mailto:jtseng@NishanSystems.com]
> > Sent: Tuesday, November 28, 2000 4:08 PM
> > To: David Robinson; Ips (E-mail)
> > Subject: RE: iFCP vs FCIP
> >
> >
> > Hi David,
> >
> > >
> > > I am no FCP expert so please correct me if I am wrong. In a pure
> > > FCP world, there is end-to-end traffic and there is
> traffic that is
> > > destined to go between AS's. The primary difference is that there
> > > is an explicit route to the border gateways in the latter
> > > case. In both
> > > the proposals, within the FCP realm the addresses are FCP
> > based until
> > > they hit an edge node.  In iFCP the destination is
> > converted to an IP
> > > address that represents the end node address (which may
> actually be
> > > a gateway back into FCP on the other side), in FCIP the request is
> > > routed to the other AS's FC border gateway and this request is
> > > encapsulated
> > > in a TCP request. Given that we are moving between AS's (I
> > > believe that
> > > is an assumption in FCIP) can we not use iFCP and instead of
> > > specifying
> > > the IP address of the end node, specify the IP address of the
> > > other AS's
> > > border gateway since FCP should already be doing some
> encapsulation
> > > to route between AS's?
> > >
> > > 	-David
> >
> > Up until recently with the creation of the DMP routing protocol, the
> > concept of AS's (Autonomous Systems, right?) was foreign to Fibre
> > Channel networking.  Most Fibre Channel networks are
> comprised of just
> > a handful of switches--the largest FC network I have ever heard of
> > being deployed is a 15 switch fabric.  Perhaps somewhere there are
> > some fabrics which are bigger, but probably not by much.
> > (Architecturally, a single Fibre Channel fabric has a
> maximum capacity
> > of 239 switches)
> >
> > FCIP does not do anything to improve the scalability limits of
> > the Fibre Channel fabric.  All it does is allow extension of the
> > FC fabric over distances using an IP network.  The FCIP gateway is
> > completely invisible and non-intrusive to the Fibre Channel switches
> > and does not change or improve the scalability or interoperability
> > limits of FC fabrics.
> >
> > On the other hand, an iFCP gateway actively participates in
> > switching and routing traffic between FC fabrics and FC devices, by
> > mapping FC addresses to IP addresses and routing them using standard
> > IP routing protocols.  Using iFCP, a storage network has the same
> > scalability limits as any other IP network (e.g., IPv4
> address space,
> > etc...).
> >
> > Josh
> >
>