[mirror-admin] rsync and updates-testing -> updates-released

[Carlos Carvalho]

Jesse Keating (jkeating at redhat.com) wrote on 21 July 2009 20:38:
 >On Tue, 2009-07-21 at 23:03 -0300, Carlos Carvalho wrote:
 >> Right now they cannot be absolute for the same reason. I meant
 >> "absolute" starting from the root of the mirror tree, of course.
 >> 
 >
 >I got push back even on that.  Doing it in repodata is a tad bit
 >dangerous, so I'd rather do it with file hardlinks.
 >
 >Why is there an aversion to hardlinks?

It's not an aversion but it's an inferior solution for the following
reasons:

- more error prone, as has happened here more than once
- less clear repository structure (admitedly subjective)
- rsync needs more memory to handle hardlinks
- increases the number of file names, thus needing more disk accesses;
  if the requests are close enough in time the (same) inode will
  already be in ram and the cost will be negligible, but if they're
  already out of cache another disk read is necessary. Without the
  large amount of hardlinks the number of names would be non-neglibly
  smaller

Ricky Zhou (ricky at fedoraproject.org) wrote on 21 July 2009 22:10:
 >On 2009-07-21 11:01:14 PM, Carlos Carvalho wrote:
 >> How about using a recent version of rsync as well??? That fossil makes
 >> our script have work around the server limitations :-(
 >Unfortunately, we don't have any control over those servers, although we
 >have been working on getting these under our control.  They're running
 >RHEL4 with rsync 2.6.3, and they're also serving content other than
 >Fedora stuff.  When we do start to run the rsync servers, they will be
 >on RHEL5, where we will have a more recent rsync version (2.6.8 right
 >now) available.

Good but not enough. rsync v3 has several improvements, some of which
are particularly useful for mirrors. One of them is a more efficient
hardlink handling, saving both memory and bandwidth at both ends.
Another is the possibility to start the download without having to
wait eons for the full file list; this visibly improves the
propagation of changes to other mirrors and thus users.

Jon Stanley (jonstanley at gmail.com) wrote on 22 July 2009 11:27:
 >In Infrastructure, RHEL is a means to an end of Fedora.  With our
 >precious little resources, we're not interested in upgrading all of
 >our systems every 6-12 months - we have more productive things that we
 >could be doing with the limited resources that we have.

It should be possible to get the latest (stable) version of rsync and
compile it on RHEL5 with minimal effort, if not directly. I do it in
Debian. Changes will be quite rare, so it shouldn't be a burden.

--