<div dir="ltr"><div>The default SSH implementation does have intrinsic performance limits, as outlined here:<br> <a href="http://www.psc.edu/index.php/hpn-ssh">http://www.psc.edu/index.php/hpn-ssh</a><br></div>so not using SSH might help with the speed, as long as the loss of security can be tolerated.<br><div><br><br clear="all"><div><div class="gmail_signature"><div dir="ltr"><div>~ Brian Mathis<br></div>@orev<br></div></div></div>
<br><br>On Tue, Oct 27, 2015 at 11:06 AM, Jim Kinney <span dir="ltr"><<a href="mailto:jim.kinney@gmail.com" target="_blank">jim.kinney@gmail.com</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><p dir="ltr">The slowdown is the encypt/decrypt. The non-server method relies on ssh for transport. You can also use rsh for no security at all and it will be faster. By using the rsync server, you drop the ssh security so if the user must enter credentials to access the NAS, you might want to double check if the rsync credentials are sent as plain text in the same way as ftp.</p><div class=""><div class="h5">
<div class="gmail_quote">On Oct 27, 2015 10:49 AM, "Todor Fassl" <<a href="mailto:fassl.tod@gmail.com" target="_blank">fassl.tod@gmail.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">I know you don't need to have an rsync server to copy files via rsync but from what I've read, rsync protocol is way fater than ssh. And you have to have an rsync server to use the rsync protocol, right?<br>
<br>
On 10/27/2015 08:46 AM, Jim Kinney wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Rsync doesn't require an rsync server. It provides a solid backup. Rsync<br>
it back and it's all golden.<br>
<br>
Tarball will need enough space to be built or will need to be built<br>
'over the wire' using a tar|<transport method>|tar process. Second<br>
optional.<br>
<br>
Tar is faster but rsync is easier.<br>
<br>
A 4TB external hard drive and sneaker net also works and provides<br>
verifiable copies. Rsync to a local drive is fast especially with an<br>
external sata port.<br>
<br>
On Oct 27, 2015 9:37 AM, "Todor Fassl" <<a href="mailto:fassl.tod@gmail.com" target="_blank">fassl.tod@gmail.com</a><br>
<mailto:<a href="mailto:fassl.tod@gmail.com" target="_blank">fassl.tod@gmail.com</a>>> wrote:<br>
<br>
One of the researchers I support wants to backup 3T of data to his<br>
space on our NAS. The data is on an HPC cluster on another network.<br>
It's not an on-going backup. He just needs to save it to our NAS<br>
while the HPC cluster is rebuilt. Then he'll need to copy it right back.<br>
<br>
There is a very stable 1G connection between the 2 networks. We have<br>
plenty of space on our NAS. What is the best way to do the caopy?<br>
Ideally, it seems we'd want to have boththe ability to restart the<br>
copy if it fails part way through and to end up with a compressed<br>
archive like a tarball. Googling around tends to suggest that it's<br>
eitehr rsync or tar. But with rsync, you wouldn't end up with a<br>
tarball. And with tar, you can't restart it in the middle. Any other<br>
ideas?<br>
Since the network connection is very stable, I am thinking of<br>
suggesting tar.<br>
<br>
tar zcvf - /datadirectory | ssh user@backup.server "cat ><br>
backupfile.tgz"<br>
<br>
If the researcher would prefer his data to be copied to our NAS as<br>
regular files, just use rsync with compression. We don't have an<br>
rsync server that is accessible to the outside world. He could use<br>
ssh with rsync but I could set up rsync if it would be worthwhile.<br>
<br>
Ideas? Suggestions?<br>
<br>
<br>
<br>
on at the far end.<br>
<br>
He is going to need to copy the data back in a few weeks. It might<br>
even be worthwhile to send it via tar without<br>
uncompressing/unarchiving it on receiving end.<br>
<br></blockquote></blockquote></div></div></div></blockquote></div><br></div></div></div>