[mirror-admin] download.wpi.edu filesystem corruption vs. not using rsync --checksum

[Kevin Fenzi]

On Mon, 26 May 2014 09:11:18 -0400
Konstantin Ryabitsev <mricon at kernel.org> wrote:

> On 25/05/14 01:56 PM, Kevin Fenzi wrote:
> > "Generating the checksums means that both sides will expend a lot of
> > disk  I/O  reading  all the data in the files in the transfer (and
> > this is prior to any reading that will be done to transfer changed
> > files), so this can slow things  down  sig‐ nificantly."
> > 
> > So, I think it's not going to be very nice for us to enable it,
> > since we already are hammered on I/O. 
> > 
> > I suppose we could try enabling it on one of our download servers
> > and see how swamped it gets next to the others? And/Or have you test
> > against it? 
> 
> Kevin:
> 
> I heartily don't recommend it. I had to do that once to rsync the full
> Ubuntu tree from mirrors1 to mirrors2 due to some file corruption on
> mirrors2, and a single run with --checksum took 38 hours, pegging IO
> on both systems in the process.
> 
> It's simply not tenable for tier-1 mirrors to offer checksum-based
> rsyncing -- the answer to "how do I fix file corruption" should really
> be "wipe and resync, as it'll probably be faster."

yeah, I was afraid of that. ;) 

ok, good to know... 

kevin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.ale.org/pipermail/mirror-admin/attachments/20140526/ef9bdbff/attachment.sig>
-------------- next part --------------
--