[mirror-admin] Please use --delay-updates

[Carlos Carvalho]

J.H. (warthog9 at kernel.org) wrote on 14 April 2010 11:24:
 >On 04/14/2010 07:18 AM, Matt_Domsch at Dell.com wrote:
 >> I received a report last night from a Fedora user in Germany who experienced a weird problem.  MirrorManager would direct him to a mirror, yum would download the repomd.xml file from it and verify it was current and correct.  Yum would then go to download the repodata/*-prestodata* file noted in repomd.xml, and would fail with a HTTP 404 (not found), and then fail over to the next mirror, only to have the same thing happen, until it hit a complete mirror.
 >> 
 >> So two requests:
 >> 
 >> 1) please use the --delay-updates option to rsync.  I know it requires more disk space, but it reduces the window in which mirrors are inconsistent like this.
 >
 >While I'll disagree that this makes sense from the mirror perspective

I think everybody agrees it's for the benefit of the user, not the
mirror :-)

 >one thing that should be imperative in using --delay-updates is the
 >use of --partial-dir=DIR.

--delay-updates implies --partial-dir=.~tmp~

 >Specifically use a directory that isn't even visible inside your
 >fedora directory,

Not sure what you mean. rsync will add an exclude rule for the partial
dir automatically. If you don't want it to appear in any way (even in
listings) you have to put it outside the repo. This means you must use
an absolute path, otherwise it'll be taken relative to the repo. Using
an absolute path is not feasible. All downloads would go to a single
directory, however there are files in Fedora with the same name in
different directories, such as bootimg.iso.

 >hopefully on the same filesystem (to help minimize i/o thrash)

Not "hopefully", it's mandatory.

In summary, do not mess with --partial-dir, just set --delay-updates.
And while you're at it, also set --delete-delay (or --delete-after if
you have a too old rsync).

 >I've attached a modified version of the kernel.org perl script that we
 >use to rsync to give people some guidance or a framework if you need it.
 >(note: the script is untested with respect to a couple of the changes)

It just clears --partial-dir, so it'll work because --delay-updates
will set it to .~tmp~ :-)

--