[ale] rsync with big files (.5GB and above)

[Jeff Lightner]

I don't think it's a generic rsync issue.   At a prior job we used to
run all day rsync scripts to transfer databases - of course that was all
done within the same VLAN.   

Perhaps if you script it so it just resumes where it left off if it
drops the connection?   

-----Original Message-----
From: ale-bounces at ale.org [mailto:ale-bounces at ale.org] On Behalf Of Greg
To: ale at ale.org
Freemyer
Sent: Monday, April 30, 2007 3:42 PM
To: Atlanta Linux Enthusiasts
Subject: [ale] rsync with big files (.5GB and above)

All,

I setup a dreamhost account like I think I mentioned before.  I've
been rsync'ing my encrypted backups there for a few weeks.

As part of my implementation process I started only sending small
files (via the --max-size option of rsync).

When I was working with files below .5 GB I had the occasional
time-out, but in general it worked most of the time.

I set my max to 2 GB about a week ago.  Since then I get a few
additional files copied each attempt, but I don't think I have sent
more than 2 files in any one rsync instance.

Does anyone know if this is a generic issue with rsync, or am I just
seeing more timeouts because my discrete rsync instances are running
longer and dreamhost is having issues.  (I've never run more than 3 or
4 hours without a timeout).

FYI: I have --timeout set to 5 minutes in my backup script, but I've
been manually running rsync during the day with timeouts as long as 2
hours.  Longer timeouts seem more reliable, but rsync still fails
after a couple of files.  And I think the timeout is per i/o, not per
file.

Thanks
Greg
-- 
Greg Freemyer
The Norcross Group
Forensics for the 21st Century
_______________________________________________
Ale mailing list
Ale at ale.org
http://www.ale.org/mailman/listinfo/ale