[mirror-admin] Please use --delay-updates

[Carlos Carvalho]

J.H. (warthog19 at eaglescrag.net) wrote on 15 April 2010 11:40:
 >I'm not saying anything against using --delay-updates explicitly, beyond
 >that it's an I/O thrash.

It's not I/O thrash. There's only one extra move for each file. It's
only a move, never a copy because rsync won't work if the partial-dir
is not in the same filesystem.

 >Well there are the problems, as I continue to watch them for a lot of
 >other distros, when you have the .~tmp~ directories in the main path,
 >specifically if an rsync is running you can end up with a large number
 >of .~tmp~ directories in your own tree.

You don't. rsync has protections for this, and even without them the
permissions are locked. So in a properly configured machine the
clients won't have access to the contents of the partial-dir, they
will at most see the dir itself:

.../fedora-enchilada/linux/development/rawhide/i386/debug[ 3:57PM] ls -ld .~tmp~
drwx------ 2 fedora fedora 4096 Apr 15 15:24 .~tmp~

 >While this is obviously
 >expected when you have, mirrors or other entities syncing from you (and
 >keep in mind there are entities that use things like FTP to sync their
 >trees) you can cause three separate and problematic occurrences:
 >	1) You can end up with a nasty nesting of .~tmp~ directories if
 >	   you have multiple servers in the chain syncing simultaneously
 >	   (again keep in mind while the mirrors are using rsync, end
 >	   users aren't all doing that)

Not true, for the reason above.

 >	2) You can end up in situations where files in .~tmp~ cannot be
 >	   deleted from your mirror.  Rsync has issues with this, why I
 >	   have no idea, but I can point out numerous instances where
 >	   if it gets synced to a client it has to be gone and manually
 >	   deleted

Not if you have a recent enough version, as everyone should. Version
2.6.8 had a bug where a temporary (not a partial) would be left. It's
a very old version though and should no longer be used.

 >	3) You get an inconsistent view across rsync, ftp and http

Irrelevant. The point here is to offer a consistent view for
downloaders. These clients will never look for anything that's
not in the distro, so they'll never see the partial-dirs.

 >So yes, some of this can be fixed with setting up appropriate excludes
 >in your rsyncd config, but this still leaves the inconsistent views
 >across ftp and http (where ftp is the only real concern).  Since Matt is
 >already suggesting people add things to their rsync scripts, I'm
 >advocating we should do full and proper atomic syncs (or as atomic as
 >makes sense).

And what makes sense is exactly what's offered by --delay-updates.
It's enough for the clients who are pulling packages, doesn't hinder
properly configured mirrors and doesn't have the absurd overhead of
the creation of an entire new tree of hardlinks at each update.

--