A Data Integrity Negotiation Scheme for iSCSI

[Mark Bakke <mark.bakke@nuspeed.com>]

Here are some thoughts on data integrity, and negotiating
an iSCSI-level scheme.  In some networks, iSCSI-level data
integrity is not a requirement, but in others, it will be
essential.  Providing an extensible mechanism to negotiate
appropriate data integrity will make it easier to bring
iSCSI into new environments, without rewriting the original
RFC.


A proposal for adding a data integrity mechanism at the iSCSI level.

This method is independent of transport-level data integrity
mechanisms, such as TCP-CRC or IPSec or other tunneling security
protocols.  It attempts to first specify an extensible mechanism that is 
independent of the actual technologies that could be used, such as 
checksums, CRCs, or hashes.  It then begins to specify an actual 
mechanism, to be used until it is deemed insufficient for some 
application.  The point is that if there is a negotiable, extensible
mechanism, future RFCs can define new mechanisms, and we don't
have to make too many predictions about the future of data
integrity right now.


Assumptions

- This method is meant to protect data from accidential
  harm, caused by undetected TCP checksum errors, or intervening
  devices that may deliver iSCSI headers and/or data between
  TCP connections.

- This method is NOT meant to be cryptographically secure.  An
  active agent could strip and (or just strip) this mechanism
  when modifying data purposefully.  This mechanism assumes that
  in cases where this may be a problem, that other security
  mechanisms, such as the transport-level mechanisms mentioned
  above, or higher-level (file or database) cryptographic
  integrity mechanisms are used.

Requirements

- Data integrity checks should be done on data separately from
  commands and headers; this allows data integrity checks on
  data to be stored in an intelligent storage device (more
  end-to-end protection), or forwarded by a gateway.

- Different integrity checks may be used on headers and commands
  vs. data.  These are negotiated at login.  Different checks
  may have different and more or less desirable properties, such
  as the ability to cover larger ranges of data, or be incrementally
  updated, or whatever.  The login mechanism allows this.

- No iSCSI-level checks are added unless they are specified at login.
  Initiators and targets not supporting data integrity checks at this
  level must interoperate with those that do.

So here's the scheme:

  1. At login, data integrity schemes are negotiated separately for
  use on data and commands.  For the purposes of
  this scheme, data is defined as the payloads in the SCSI Data
  Request and Response.  Commands are defined as everything else,
  including iSCSI headers.

  2. Most schemes will add the command integrity check of their 
  choice to:

  - The end of an iSCSI non-data request or response (command
    integrity check).
  - Immediately after the iSCSI header in a data request or response,
    if length > 0.
  - Immediately after the iSCSI header in a Command Request if the
    Immediate (I) bit is set.

  3. Most schemes would add the data integrity check of their
  choice to:

  - The end of the data for an Immediate Command Request, Data
    Request, or Data Response.

  - OR -

  - After Every xxxx bytes (perhaps 8k) of data in an Immediate
    Command Request, Data Request, or Data Response.

  - OR -

  - Both of the above.

Granularity

  Some checksum schemes may weaken significantly if used on
  continuous data of over a certain size.  These schemes should
  define a granularity, or data length, after which individual
  integrity checks are inserted.

Login Request

  If an initiator wishes to use an iSCSI-level data integrity scheme,
  it will include one or both of the following fields:

dataIntegrity: <scheme-name> ...
commandIntegrity: <scheme-name> ...

  dataIntegrity specifies the scheme names, in order of preference,
  that the initiator would like to support.  The target must choose
  the highest-preference scheme that it wishes to allow.

  commandIntegrity specifies scheme names it would like to use
  for commands and headers.

  If unspecified, the default for both values is "none".

  Example:

dataIntegrity: CRC32
commandIntegrity: Checksum32 none

  This specifies that the initiator will only accept the CRC32
  scheme for data, and would prefer the Checksum32 for commands,
  but will accept no data integrity for commands if Checksum32
  is not available.

Login Response

  When a target receives a login request, it checks to see if
  it supports, and is willing to support, one of the schemes
  requested by the initiator.  If any are supported, it returns
  the one most preferred by the initiator.  If none are supported,
  it fails the login with a new error code.  Data and command
  integrity schemes are negotiated separately.

  Note that even if a scheme, such as "none" is supported by
  a target, it may refuse to use it for a particular target,
  due to the user configuring the iSCSI target in a more
  protective or paranoid manner.

  Example:

dataIntegrity: CRC32
commandIntegrity: none

  The login response is not protected by the negotiated command
  integrity check.  All subsequent requests from the host are


The CRC32 Scheme

  Other than "none", "CRC32" is the scheme specified in this
  document.  Other schemes that may be more applicable should
  be proposed as well.

  If the CRC32 scheme is negotiated for headers, a 32-bit CRC
  is at the end of an iSCSI request or response, or between
  the request and its data, as described above.

  If the CRC32 scheme is negotiated for data, a 32-bit CRC
  is added to the end of all data, covering ONLY the data (headers
  and commands sold separately).
  
  The big open question here is whether to support a CRC at the
  end of the data request, or at the end of every [8k?] "block"
  of data, or both.  The advantage of a CRC over just the block
  is simplicity; this works fine for most block device accesses,
  since file systems and databases usually write in 2, 4, or 8k
  clusters.  However, if the data is of sufficient size as to make
  the check too weak (tape backup software and specialized file
  systems for large block data can write 64k .. 1MB or more at
  a time), we would need to add the CRC more often.

  [ WORK - Insert the appropriate algorithm or polynomial here ]

  There's more to do here, but I wanted to at least float something
  out for comments, and I think that the negotiation scheme is
  more important than nailing down the exact scheme right now,
  especially since data integrity requirements will change
  depending upon the end user's application and network.

Notes

  All of the above schemes negotiate insertion of commands and
  data outside the normal headers and data, and therefore do not
  specify modifications to either.

  If iSCSI is used entirely over a layer-2 network (no l3 routers
  in the middle), the data-link layer CRC is kept end-to-end,
  and is not regenerated by bridges or switches.  iSCSI transfers,
  regardless of their size, are broken up into 576, 1500, or at
  worst 9k (GbE Jumbo Frame) packets, each with their own 32-bit
  CRC.  In this case, the only likely place for data integrity
  problems is within the iSCSI host and device, which will have
  the exact same characteristics as if Fibre Channel, parallel
  SCSI, or any other storage interface were used.  The strength
  of the TCP checksum in this case should not be an issue.


-- 
Mark A. Bakke
NuSpeed, Inc.
mark.bakke@nuspeed.com
763.398.1054