[ale] has anyone successfully used RSYNC to....

[Courtney Thomas]

On 03/03/12 15:03, Wolf Halton wrote:
> On Fri, Mar 2, 2012 at 2:23 PM, Ed Cashin<ecashin at noserose.net>  wrote:
>> I've done that, but more often, "NEWdrive" is bigger than "HD1" and
>> "HD2".  I don't have the exact commands handy, but I can describe
>> them, and whatever anybody tells you about this, make sure you
>> actually do the procedure you choose *before* you're forced to.
>> There's inevitably some learning and tweaking, even if you get recipes
>> from other people.
>>
>> So in general, for rsync, I used "-a" plus the option that preserves
>> sparse files ("-S") and the one ("-H") that preserves hard links, and
>> often the one ("-c") that uses checksums to ensure data integrity.  Of
>> course, because rsync works over ssh, it's easiest to keep HD2 in a
>> separate machine, so that you can rsync over the network.  I use "-x"
>> to stay on one filesystem instead of going into stuff like /proc.
>>
>> Some UNTESTED illustrative example commands from memory are below,
>> assuming the only real filesystems on the localhost are / and /home.
>>
>>   rsync -avSHx / otherhost:/backup-of-hd1
>>   rsync -avSHx /home/ otherhost:/backup-of-hd1/home
>>
>> The most tricky thing about rsync is probably that trailing slash in
>> the second command.  It means "the contents of".  Without it, rsync
>> would make a new directory called "home" inside /backup-of-hd1/home on
>> otherhost.
>>
>> For the boot sector, I would copy it to a local file before the rsync
>> with dd.  But I no longer think that's as good as getting very
>> comfortable with the installation of a boot loader like grub or lilo.
>> There's rarely information in the boot sector that cannot be replaced
>> with something just as good when you're moving onto NEWdrive.  Most of
>> the time you can live boot a CD and run grub or something to install
>> the boot loader in the MBR of NEWdrive.
>>
>> For the partition table, I used to use sfdisk to dump the partition
>> info to a format that sfdisk can read to recreate the same
>> partitioning scheme.  Now I don't think that's necessary either,
>> because if you're doing a high-level file-based backup like rsync
>> does, you can just partition NEWdrive however you want, and it's
>> probably going to be bigger anyway, so good.  And maybe you've learned
>> something and NEWdrive will have a better partitioning scheme than HD1
>> did.
>>
>> For formatting, I think you mean filesystem creation.  In that case,
>> again, maybe there's a better choice for the filesystem on NEWdrive
>> than the one you were using on HD1, so it depends.  Because the backup
>> is at the file level, you can choose whichever filesystem is best for
>> your needs.  For example, maybe you have 3 partitions on MYNEWDRIVE
>> and want ext3, so you could do something like "mkfs -t ext3
>> /dev/MYNEWDRIVE1" and "mkswap /dev/MYNEWDRIVE2" and "mkfs -t ext3
>> /dev/MYNEWDRIVE3".
>>
>> After that, mount that new filesystem use the same rsync options to
>> copy the files from the mounted filesystem of HD2 to the new
>> filesystem on NEWdrive.  Something like this is what you'd do on
>> "otherhost" after those filesystem creation commands:
>>
>>   mkdir /mnt/NEWdrive
>>   mount /dev/MYNEWDRIVE1 /mnt/NEWdrive
>>   mount /dev/MYNEWDRIVE3 /mnt/NEWdrive/home
>>   rsync -avSHx --progress /backup-of-hd1/ /mnt/NEWdrive
>>   vi /mnt/NEWdrive/etc/fstab  # if you've changed the partitioning
>> scheme or filesystems
>>
>> These commands are pretty darn close, but they're untested and
>> provided without warranty.  If anyone sees errors, please chime in.
>>
>> OP, please practice using them inside virtual machines or something
>> until you're as comfortable as anyone else if you intend to do this
>> when other people are counting on it working successfully.
>>
>> On Fri, Mar 2, 2012 at 1:38 PM, Courtney Thomas
>> <courtneycthomas at bellsouth.net>  wrote:
>>> ...(1) back up all their filesystems from HD1 to an archive hard drive HD2,
>>>
>>> (2) then replaced HD1 with a NEWdrive which was partitioned, labeled and
>>> formatted exactly duplicating the replaced HD1, and then
>>>
>>> (3) RSYNCed the archived files on HD2 to the NEWdrive, with the result that:
>>>
>>> the NEWdrive was not only without fault, but that the previous
>>> functionality
>>> of HD1 was exactly duplicated ?
>>>
>>> If yes to all 3, would anyone care to share
>>>
>>> (1) the RSYNC backup command for HD1 ->  HD2
>>> (2) their partition, label and format commands for NEWdrive
>>> (3) the RSYNC restore command for HD2 ->  NEWdrive
>>>
>>> This is obviously of overwhelming importance to all and if anyone can
>>> point to a
>>> reliable source, or provide it outright themself, for the above sought
>>> command
>>> summary, my gratitude would be boundless  :-)
>>>
>>> I am aware that rsync, fdisk, e2label and mkfs are all documented commands,
>>> but a successful usage, as herein outlined, is, at least in my case,
>>> frustrating
>>> days away of trial and error, having already wasting such time in
>>> pursuit of such
>>> an outcome using 'dump'.
>>>
>>> Sincerely,
>>>
>>> Courtney
>>> _______________________________________________
>>> Ale mailing list
>>> Ale at ale.org
>>> http://mail.ale.org/mailman/listinfo/ale
>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>> http://mail.ale.org/mailman/listinfo
>>
>>
>> --
>>    Ed Cashin<ecashin at noserose.net>
>>    http://noserose.net/e/
>>    http://www.coraid.com/
>>
>> _______________________________________________
>> Ale mailing list
>> Ale at ale.org
>> http://mail.ale.org/mailman/listinfo/ale
>> See JOBS, ANNOUNCE and SCHOOLS lists at
>> http://mail.ale.org/mailman/listinfo
> I am using rsync to back up zipped copies of configs and whatever else
> is needed.  This is not a 1:1 of the whole originating disc
> partitions, as you say you want - but just the important pieces.
> Presuming the original drive doesn't fail, but only some piece of the
> puzzle failed, it is still a manual task to recover the piece you
> need.  My thinking is that if the original drive fails, I have either
> ZRAID or a system snapshot to return me to the original OS state, and
> the only thing I am missing is the recent data.
>
> tar xzf thearchive.tar.gz
> rsync -avz the-directory-you-need  root at yourserver.com:/path-towhere-it-goes/
>
> This is one of the most common errors with rsync as well, in that it
> is a basic reversal of the original rsync command. B ->  A rather than
> A ->  B
> If you are going to be running your rsync from a 3rd machine, as you
> would have to be doing if you are replacing whole disc-images, I
> think, you would have full paths at both ends of your command.
>
> Ed ++
> Always test major file-moving on VMs before you trust the process on
> your production machines.
>
> Wolf
>
Wolf,

Thank you for your response.

Having not used rsync, why are 3 machines necessary to make whole disk 
copies ?

Why can't an unused disk on the syncing machine be sync'd with the 
desired disk, wherever it's located, including on the syncing machine ?

Cordially,

Courtney