[mirror-admin] rsync and updates-testing -> updates-released

[Axel Thimm]

Hi,

currently rsync does not detect file relocations/renames like the
updates-testing to updates(-released) moves and while there are a
couple of patches floating around (for local renames only) it is
unlikely that there will soon be an rsync that does.

So when a package moves from updates-testing to updates, rsync
considers this to be a new file and pulls it again, every package is
therefore pulled by mirrors twice.

If instead of moving the files they were linked, then rsync -h would
notice and wouldn't pull it again. The drawback is that the old file
would have to remain in updates-testing.

Could we use a link-and-expire method instead of directly moving from
testing->updates? If we assume the mirrors sync at least once a day,
an expiry of 1 1/2 days for example would save half the bandwidth (of
packages that go testing first, not security updates or
rawhide). Since the "old" files are hardlinked they won't be consuming
more than an extra inode.

Alternatively if this is unlikely to happen there could be a way in
between:

   Hardlink the contents of updates-testing and updates-released into
   a common dot-named folder. When packages move from updates-testing
   to updates-released, rsyncs that include this folder will identify
   that the "new" package is a hardlink to a package in this dot-folder
   and simply hardlink it.

This method avoids having to set up expiry methods and would work out
of the box with most mirrors already. It will consume inodes only but
will cut the bandwidth of all packages moving through testing by two.
-- 
Axel.Thimm at ATrpms.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
Url : http://mail.ale.org/pipermail/mirror-admin/attachments/20090715/e46a54cf/attachment.bin 
-------------- next part --------------
--