[mirror-admin] Mirror Prep Consensus?

[Chuck Anderson]

On Mon, Sep 08, 2008 at 04:06:11PM -0700, C.M. Connelly wrote:
> 
> Is there a consensus on what preparatory work mirror admins could
> do to make syncing the new content easier?  In particular, it
> sounds like creating local copies of some directories prior to
> syncing would be A Good Thing (tm), but it isn't entirely clear to
> me from the messages and poking around on my (enchilada) mirror
> exactly which directories should be duplicated to make everything
> work properly.

Note that my earlier suggestion of using hardlinks won't work, since 
the server side won't have the hardlinks, leading to oscillating 
downloads as every other download overwrites the "other" hardlinked 
copy on your end.  So you have to use regular copies.  I found this 
out the hard way.  There might be a special rsync option that would 
cause it to "break" the hardlinks locally if the server side files 
were not harlinked and were different, but I didn't pursue that 
option.

Run this before rsync:

cd fedora/linux/updates
cp -a 8/i386 8/i386.newkey
cp -a 8/ppc 8/ppc.newkey
cp -a 8/ppc64 8/ppc64.newkey
cp -a 8/SRPMS 8/SRPMS.newkey
cp -a 8/x86_64 8/x86_64.newkey
cp -a 9/i386 9/i386.newkey
cp -a 9/ppc 9/ppc.newkey
cp -a 9/ppc64 9/ppc64.newkey
cp -a 9/SRPMS 9/SRPMS.newkey
cp -a 9/x86_64 9/x86_64.newkey
cp -a testing/8/i386 testing/8/i386.newkey
cp -a testing/8/ppc testing/8/ppc.newkey
cp -a testing/8/ppc64 testing/8/ppc64.newkey
cp -a testing/8/SRPMS testing/8/SRPMS.newkey
cp -a testing/8/x86_64 testing/8/x86_64.newkey
cp -a testing/9/i386 testing/9/i386.newkey
cp -a testing/9/ppc testing/9/ppc.newkey
cp -a testing/9/ppc64 testing/9/ppc64.newkey
cp -a testing/9/SRPMS testing/9/SRPMS.newkey
cp -a testing/9/x86_64 testing/9/x86_64.newkey

You might want to make the copied files unreadable to your downstream 
rsync clients while you are rsyncing so they won't accidentally 
download the exact copies until your own sync is complete.  E.g. if 
you sync from a different user account than your rsync server runs as 
(recommended):

find .  -name '*.newkey' -ls -exec chmod 700 {} \;

Unfortunately, I've found that each dir will get flipped back to 755 
as soon as your rsync process starts downloading that dir.  You could 
run the above command periodically during the rsync to reset it back 
to 700 and then open up the dirs after the sync is complete:

find .  -name '*.newkey' -ls -exec chmod 755 {} \;

--