[ale] Tape RAID?

[Stephan Uphoff]

James P. Kinney III wrote:
> Since the write to tape is slow compared to write to HD, is it feasible
> to make a tape RAID process? I have seen several similar things in the
> enterprise realm but they were not quite what I'm envisioning. They
> would do multiple stream backups. All of one file would be on a single
> tape. The next file might be on a different tape.
> 
> I'm thinking an evenly split data/parity spread across the array of tape
> drives. This would provide all the joys of RAID for a low cost/GB backup
> system. The aim is to have a tape system that can accept a data stream
> as fast as the hard drives can deliver it.
> 
> This would be a real PIA to do using different tape drives that write at
> different speeds. But if the drives all are equivalent, the RAID kernel
> code can be reused to split up the data stream and then a hack on the
> delivery part to support the tape parameters.
> 
> Feasible? Bad idea? Been done already? Jim should not try to think
> before the second cup of coffee has been fully ingested?

You might want to look at (old?) IEEE mass storage conference proceedings.
Tape set scheduling to combine multiple tapes for striping was a 
constant theme.

Are you sure you get full speed out of a single drive right now?
Good buffering can improve speeds considerable.
Sometimes writing a backup image to a staging drive first also helps.

While a HD is faster than a Tape - seeking or complex operations needed for
a backup (checksum of large files) can cause huge changes in data rate.
Normally the backup will wait for the tape - but in between the tape will empty
all data buffers. Then the tape has to stop, wind back a bit, speed up ...
Not good for speed or the tape media.

This looks like something to implement in user space as a multi-threaded 
utility
to me.

It might also a good idea to write a test utility first that consumes data at 
a configurable
rate and sends it to /dev/null and counts the number buffer underruns.
With the test utility you can check the cost/time benefit of different 
solutions.

	Stephan