Re: Calling for Tape & Backup Application Experts

[Stephen Bailey <steph@cs.uchicago.edu>]

Somesh,

> If you are out there on the list and can help with creating a model
> of the backup app, that will help resolve some of the issues.

I expect I'm considered to be a biased observer, but the only backup
app that I've ever used that worked decently is amanda.

It's relatively configurable, but the typical configuration (the only
one I've seen used, at several, independent sites) is:

  1) backup (e.g. dump) streams from client to holding disk on backup server
  2) server streams data from holding disk to tape (prevents underruns) 
  3) if tape error occurs, notify operator and retain backup data on
     the holding disk
     if backup disk is full, stop backing up clients to holding disk
  
It also does all sorts of complicated stuff to chose among backup
levels, but that's not really germane here, except that it will allow
you to back up a massive storage foot print with a relatively small
number of tape drives and relatively small holding disk.

The upshot of this is that amanda uses programs like tar, dump and
cpio as primitives in which errors are expected to occur occasionally,
and recovery is required.  It is a mistake to rely solely upon simple
backup programs that have no operator intervention loop in them.

In my experience (I was an operator at one such site), a very common
reason why the backups failed was because high-density cartridge tape
drives require cleaning after a relatively small number of hours, and
(at least) Exabyte drives would not permit a tape to be used when a
cleaning was due.  It did give some advanced warning, like flashing
lights, but you know how easy it is to miss a flashing LED in a
machine room (they flash when the tape is operating too!), and with
the small interval between cleanings (when the tape drive is running
near 100% duty cycle), it seemed like it was ALWAYS in need of
cleaning.

I haven't used a Windows backup apps, but I can't help thinking the
Amanda approach is not novel for a serious backup app (as opposed to
NT backup or other `consumer grade' apps).

While we're gathering requirements, backup is only one of the two
major applications of tape.  The other is streaming data logging, also
known as the `telemetry problem'.  In this case, data is running to
the tape almost continuously, and an operation failure may not be a
matter of trying again tomorrow.  Then again, it might.  There's
usually a large direct access device (e.g. RAID box) used as a buffer
to the tape, but I'm not sure if it's sized to allow for operator
recovery or what.  I don't see how any other solution would be
possible.

Steph