<p dir="ltr">Rsync doesn't require an rsync server. It provides a solid backup. Rsync it back and it's all golden.</p>
<p dir="ltr">Tarball will need enough space to be built or will need to be built 'over the wire' using a tar|<transport method>|tar process. Second optional. </p>
<p dir="ltr">Tar is faster but rsync is easier.</p>
<p dir="ltr">A 4TB external hard drive and sneaker net also works and provides verifiable copies. Rsync to a local drive is fast especially with an external sata port.</p>
<div class="gmail_quote">On Oct 27, 2015 9:37 AM, "Todor Fassl" <<a href="mailto:fassl.tod@gmail.com">fassl.tod@gmail.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">One of the researchers I support wants to backup 3T of data to his space on our NAS. The data is on an HPC cluster on another network. It's not an on-going backup. He just needs to save it to our NAS while the HPC cluster is rebuilt. Then he'll need to copy it right back.<br>
<br>
There is a very stable 1G connection between the 2 networks. We have plenty of space on our NAS. What is the best way to do the caopy? Ideally, it seems we'd want to have boththe ability to restart the copy if it fails part way through and to end up with a compressed archive like a tarball. Googling around tends to suggest that it's eitehr rsync or tar. But with rsync, you wouldn't end up with a tarball. And with tar, you can't restart it in the middle. Any other ideas?<br>
Since the network connection is very stable, I am thinking of suggesting tar.<br>
<br>
tar zcvf - /datadirectory | ssh user@backup.server "cat > backupfile.tgz"<br>
<br>
If the researcher would prefer his data to be copied to our NAS as regular files, just use rsync with compression. We don't have an rsync server that is accessible to the outside world. He could use ssh with rsync but I could set up rsync if it would be worthwhile.<br>
<br>
Ideas? Suggestions?<br>
<br>
<br>
<br>
on at the far end.<br>
<br>
He is going to need to copy the data back in a few weeks. It might even be worthwhile to send it via tar without uncompressing/unarchiving it on receiving end.<br>
<br>
<br>
<br>
_______________________________________________<br>
Ale mailing list<br>
<a href="mailto:Ale@ale.org" target="_blank">Ale@ale.org</a><br>
<a href="http://mail.ale.org/mailman/listinfo/ale" rel="noreferrer" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
See JOBS, ANNOUNCE and SCHOOLS lists at<br>
<a href="http://mail.ale.org/mailman/listinfo" rel="noreferrer" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
</blockquote></div>