[ale] duplicating a working system for backup.

[DJ-Pfulio]

TL;DR
Use ddrescue instead of dd for whole drive replication.  It is dd, but with progress and when a problem is encountered, it will skip that sector and continue getting most of the easy data first, then it comes back and tries to get the harder data.

Cloning a GPT disk to a different sized HDD fails to put the 2nd copy of the partition table where it belongs, so you'll want to fix that.

Don't think I'd run an OS off any HDD over 500 MB in size. After all, the OS only needs 25G or 35G if it is bloated, right?  Keep the data and OS separate. Makes for less risky reinstalls, upgrades, and smarter backups.

If you can't restore everything except huge amounts of data in 30-45 minutes, I'd say your backup & restore processes need work. rsync with just a mirror isn't really enough backup tech for me.  Why not have 90 days of daily versioned backups for about 1.20x the original storage using a command that is almost the same as the rsync command and solves a number of issues that rsync fails to solve for backups?

Fixing UUIDs post-clone is easy. There's a command for that, but we have to remember to run it. If you don't use advanced volume management, then you'll need to fix the fstab too. If you use LVM, just rename the LVs to have "backup".  Heck, if you use LVM, use the pvmove command to clone the partitions while the system keeps running.

Would love to see how other people handle their versioned backups efficiently. We can put up a repo with a few different scripts, if someone is interested enough to show up and ask at a meeting. After all, these scripts all have to be customized for the systems and the backup storage anyway.

IMHO.

On 8/12/20 4:07 PM, Scott Castaline via Ale wrote:
> My system is currently running Fedora 32 on a 8 TB hard drive. My
> original backup drive went on a four foot dive to a hard floor. To
> confirm the damage I powered it up and my wife could hear the
> screeching sound it made downstairs. I have since bought a 10 TB HDD
> and originally hooked it up to my PC and booted from DVD and did a dd
> if=/dev/sda of=/dev/sdb. After 5 days it was still running and as in
> my rush to just get started I didn't take time to look up means of
> monitoring status so I thought maybe it was hung even though it
> looked like it was still working under ps -ef.
> 
> The original drive is divided up into 3 partitions with part 1
> starting at sector 2048, I don't remember why, I just remember that
> it was necessary for gpt drives. part 1 is set for 1Meg or 2048
> sectors and is the BIOS Boot, part 2 is my /boot for Linux @ 1.2G and
> part 3 is the rest of the drive fully encrypted with dmcrypt and this
> contains the rest of the filesystems (/, /home, & /pub) I have
> several image files of CDs and DVDs which is my music and video
> libraries as well as various linux distributions and many PDFs. I
> assume that the delays were on the image files as when I went ahead
> and pulled the plug and rebooted the new drive had some serious file
> system issues. Once fsck did it's thing I wound with about 5 TB of
> lost files. Then I started having issues bouncing back and forth
> between the 2 hdds due to the UUIDs being the same which I had
> forgotten about.
> 
> So I wound up removing the new drive completely and rebooted several
> times (both cold and warm boots) with the original drive and
> everything seemed okay with dev/sda again. I have since reconnected
> the new drive after totally disconnecting the main drive and booting
> from DVD I did dd if=/dev/urando of=/dev/sda (which was the 10TB HDD
> at that time). So now I'm back at square one. I want to backup my
> whole system onto the 10TB HDD and if need be, be able to swap out
> the drive and pick up from the last backup. I normally use rsync for
> my weekly backups using Flash Drives and a HDD. What's the best and
> quickest way to do this, to have a totally bootable working system on
> my backup HDD?
> 
> Thanks for any advice
>