[mirror-admin] master server sync stats and recommendations

[Axel Thimm]

On Wed, Apr 22, 2009 at 08:39:57AM -0500, Chris Adams wrote:
> Once upon a time, Axel Thimm <Axel.Thimm at ATrpms.net> said:
> > Aside from buggy null-updates from the server side, what if you want a
> > pure rsync solution, do we want to start mixing protocols?
> > it more granular helps as well, unless we only target fedora-enchilada
> > consumers.
> 
> Well, if you are going to look at the content of a "last updated" file,
> or compare its timestamp to something, your aren't pure rsync already
> (you'll have to have some scripting, be it shell, perl, python, etc.).
> Also, since you should have some type of locking to prevent overlapping
> rsyncs, you should have some type of scripting already.

With pure rsync I meant protocol-wise, of course we all wrap our
scripts around it one way or the other.

But as a minimal rsync & sh implementation one could have

rsync rsync://master/module/.lastupdated /tmp/somefile
diff /tmp/somefile /storage/my/local/mirror/.lastupdated > /dev/null \
  && rm -f /tmp/somefile && exit
<Full rsync command>

> As far as fetching the timestamp via HTTP HEAD: well, we are HTTP
> mirrors after all.

Which doesn't mean that the have any http clients on them, in fact
hardening an exposed server sometimes means removing all http/ftp
clients.

> I do think it would be reasonable to have a fullfilelist at the top of
> each rsync module (even though the modules are nested).

What's the use of fullfilelist, if only its date is used? Why not
simply a one-liner containing the date itself, so that even human
inspection can quickly gather wheather this mirror is stale or not.

Actually the mirrorcrawler could even rely itself on this file on
mirrors to judge how outdated they are.
-- 
Axel.Thimm at ATrpms.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
Url : http://mail.ale.org/pipermail/mirror-admin/attachments/20090422/6f3e9842/attachment.bin 
-------------- next part --------------
--