[ale] best way to copy 3Tb of data

[Jim Kinney]

Rsync doesn't require an rsync server. It provides a solid backup. Rsync it
back and it's all golden.

Tarball will need enough space to be built or will need to be built 'over
the wire' using a tar|<transport method>|tar process. Second optional.

Tar is faster but rsync is easier.

A 4TB external hard drive and sneaker net also works and provides
verifiable copies. Rsync to a local drive is fast especially with an
external sata port.
On Oct 27, 2015 9:37 AM, "Todor Fassl" <fassl.tod at gmail.com> wrote:

> One of the researchers I support wants to backup 3T of data to his space
> on our NAS. The data is on an HPC cluster on another network. It's not an
> on-going backup. He just needs to save it to our NAS while the HPC cluster
> is rebuilt. Then he'll need to copy it right back.
>
> There is a very stable 1G connection between the 2 networks. We have
> plenty of space on our NAS. What is the best way to do the caopy? Ideally,
> it seems we'd want to have boththe ability to restart the copy if it fails
> part way through and to end up with a compressed archive like a tarball.
> Googling around tends to suggest that it's eitehr rsync or tar. But with
> rsync, you wouldn't end up with a tarball. And with tar, you can't restart
> it in the middle. Any other ideas?
> Since the network connection is very stable, I am thinking of suggesting
> tar.
>
> tar zcvf - /datadirectory | ssh user at backup.server "cat > backupfile.tgz"
>
> If the researcher would prefer his data to be copied to our NAS as regular
> files, just use rsync with compression. We don't have an rsync server that
> is accessible to the outside world. He could use ssh with rsync but I could
> set up rsync if it would be worthwhile.
>
> Ideas? Suggestions?
>
>
>
> on at the far end.
>
> He is going to need to copy the data back in a few weeks. It might even be
> worthwhile to send it via tar without uncompressing/unarchiving it on
> receiving end.
>
>
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20151027/901f6d20/attachment.html>