[ale] Hard Drive Failures

[JD]

Inline with lots-o-snipping ...

On 02/21/2013 08:04 PM, Ron Frazier (ALE) wrote:
> Agree mostly.  Disagree slightly.

Completely understandable.

> What I will lose is a good amount of sleep and hassle.  I far and away prefer
> to have image backups versus any other kind.  If the drive did puke, and I
> didn't have a recent image, it would take me probably a week to reinstall the
> os, reinstall all the apps, configure all the apps, install plugins in the
> apps, configure all the plugins on all the apps, and do all the system tweaks
> that I want to make the machine run the way I want.  It would probably take
> me another few days to get all my email restored, and rebuild all my filters
> to filter 10's of thousands of messages.

If you do a backup correctly, it isn't a week to restore, but 15 minutes for a
small box.  If it is a VM, less.  These are not image-based backups either.  To
me, after critical data, settings are the most important things to backup AND
restore. Using current Linux backups, it is fairly simple to backup everything
and restore it.  Plus, because it is not extremely hardware sensitive, almost
any machine can be used for the restore.

> On the other hand, let's say I had cloned the hdd last night.  I swap in the
> cloned drive, and I'm literally back up and running within 15 minutes.  I
> just restore the latest data from my online backup, download any recent
> email, and re do anything else I remember that occurred since the last online
> backup up to 6 hours ago.

I've cloned hard drives and still do. well, not really, but I do get a
compressed image that can be restored.

The main issue with cloning is that it loses backup versioning. We end up with a
mirror and it is HUGE, compared to what actually changed.

rdiff-backup supports versioned backups.  The first, is a mirror.  From that
point on, only changed files are moved over into the "mirror" area. Any parts of
files that are replaced during that process are moved into an archived area
permissions captured and gzipped. Notice, that I said parts of files.  The
latest backup is always a mirror, so restoring 1 file is just a cp.  File
permissions (and ACLs with an addon) are retained across systems.  Diffs are
retained in a highly efficient manner.  Depending on the change rate and size of
your data, only 10-20% more storage is used for 30-60 days of versioned backups
that required for the mirror. It is amazing.

Basically, if a virus were to get onto the backup system through backup
processing, I'd have 30 to 60 days to realize it and would see the day that the
file changed.

Plus all my settings are safe.  Probably under 10 seconds to restore ~/.config/

> Relatively simple and painless.  If I can use something like Spinrite to
> recover the drive, even if it's just to clone it to a replacement drive.
> Then I don't lose any data or have to do any reconfiguration.

What if your clone has the virus you didn't realize that you got 3 weeks ago?
Or simply an important file that you've been working on off and on over 3 weeks
became corrupt?

> The problem comes into play because I don't clone my drive every night.  I
> don't like to leave the backup media attached, since a virus or electrical
> problem could take it out.  So, I prefer to attach the backup media only when
> I clone the drive.  Also, I have to reboot the machine and boot from a CD to
> do the imaging, then reboot the machine to use it again.

Backups need to be 100% automatic or they are not done.  I used to do manual
backups. After about 18 months, that slowed to monthly manual backups, then
annual .... which is next to worthless.  The data far outgrew my ability to back
it up, then a RAID0 set had 1 HDD fail.  I lost 80% of my data, due to my own
foolishness.  These days, I don't add storage without adding backup storage too.

> The end result is that my image backups don't get done too often,
> particularly with 4 PC's.  So, if I could find a way to automatically clone
> each PC's hard drive every night, or at least every week, and automatically
> detach and shut down the backup media when done, and reattach it when the
> next clone job is due, then I could be in a position to really not care too
> much if a drive fails.  If I could get snapshot versions like JD has
> mentioned elsewhere in this thread, so much the better.  My number one goal
> for my backup is to enable me to restore a complete system, including all
> data, all applications, and all settings, either within 15 minutes, or at
> most within a few hours.

For cloning PC OSes, check out partimage. It can write over the network, but
because it does an image, it does need to be booted outside the normal OS. A
500MB partition just for imaging could make a lot of sense, rather than having a
USB drive that needs to be moved around.  Data should be backed up more
efficiently. IMHO. Images need to be minimal and only have a place for
MS-Windows. On Linux, it simply is not necessary.  librsync is fantastic and
most Linux-based backup tools use it.

30 minutes to image after booting a different OS
 or
2 minutes to backup all the data while leaving the machine running.
You can pick.

Backup tools have come a long way in the last 2 years.  Duplicity (and the 10
GUIs built on top of it) have brought the "best practices" to a home user.

Bacula is impressive, but it is definitely an enterprise-class tool with
enterprise-class complexity.

BTW, I know that most of my backups work.  I've moved machines for about 7
machines recently. Basically, I just did a backup, shutdown 1 box, brought up
the other and restored.

Isn't that how it should work?