[mirror-admin] "file has vanished" again

[Rob van Nieuwkerk]

On Fri, 16 Oct 2009 14:05:34 -0500 (CDT)
Mike McGrath <mmcgrath at redhat.com> wrote:

> On Fri, 16 Oct 2009, Chris Adams wrote:
> 
> > Once upon a time, Mike McGrath <mmcgrath at redhat.com> said:
> > > Do you see it with mirroring other distributions?
> >
> > The only other major thing I mirror is CPAN, and I don't see it there.
> >
> 
> It seems impossible to avoid this situation completely since you'll always
> get a file list at one point in time, and finish syncing at a later point
> in time.  During that period, if a file is deleted, rsync will complain.
> If that thought process is wrong, please correct me, my rsync-foo is good
> but not expert.

Of course, that's a thing that one would expect to happen.
But the weird thing here is that this situation can STAY for many
hours (or even days !) on the same file(s) with different rsync runs.

Seems that something is lying to rsync about the files available
when it scans for the list.  When it comes to the point of syncing
the actual files they are not there.

I've always suspected it to have something to do with (incorrectly)
caching things on load-balancing equipment or NFS-setup.  Or nodes
in a load-balancing cluster that are out of sync.  Or nodes that are
ping-ponging between old & new content because of incorrect syncing-loops.

Do you know if your rsync server is maybe adapted (for performance reasons)
to fetch it's file-list from a different location (so not every rsync run
forces a scan over the whole tree) ?

	rob

--