[mirror-admin] ERROR: chroot failed for fedora-web

[Carlos Carvalho]

Matt Domsch (Matt_Domsch at dell.com) wrote on 15 January 2009 20:06:
 >If people are willing to run a (gasp: python!) script on their servers
 >(hey, you already run report_mirror...)

I don't anymore, exactly because it's too seeky, and also because now
we update more often. I still run it from time to time, when the load
is smaller.

I was going to ask you for the specs of report_mirror, that is, what
info it sends and how. Then I can provide you the info with what I
have here without the dreadful disk scan.

 >Master: os.walk() and generate a list of dirs and files with the
 >(permissions,size,timestamp) tuple.  Hey look - report_mirror does
 >this already...  Write this out as a pickle.
 >
 >Downstream Mirrors:  rsync pickle.  If it hasn't changed, exit.  If it
 >has changed, read the pickle, do your own os.walk(),

Not necessary to do os.walk.

 >drop them into a python set() object, and diff them. Regen the file
 >list from that. Feed list to rsync.

That's not enough...

 >This will reduce the load on the upstream mirrors considerably, at the
 >cost of needing to run a script on the downstream mirrors, the results
 >of which are fed to rsync.

Yes. And with the right script downstream also wins because it doesn't
do the local scanning.

--