[mirror-admin] ideas for recognizing bitflips / dir changess

[Ask Bjørn Hansen]

On May 2, 2009, at 22:34, Matt Domsch wrote:

> the question becomes: would this, or something like it, be valuable?
> Would you use it?  Would you prefer a different form than an RSS feed
> for such?  We would need a client-side parser application, which ran
> rsync against the specific recently-changed directories, so similar to
> the rsyncFilter (in fact, in my quick hack it uses the same SQL
> query).  But you could execute it every 30 minutes whereas you
> couldn't do the same for rsync sanely.

For CPAN what we've started testing is a process that syncs a metadata  
file with the latest updates very frequently.

	http://search.cpan.org/dist/File-Rsync-Mirror-Recent/

CPAN has much less data in GB, but I suspect a comparable number of  
files (~350k for CPAN; I'm not sure how many files a full Fedora  
mirror is).


  - ask

--