[mirror-admin] master server sync stats and recommendations

[Simon Valiquette]

Axel Thimm un jour écrivit:
> On Wed, Apr 22, 2009 at 10:41:28AM -0500, Matt_Domsch at Dell.com wrote:
>> http://www.debian.org/mirror/push_mirroring
>> describes how Debian does push mirroring, complete with the ssh setup.
>> I've not set this up before, but we've discussed here before using the
>> same setup to trigger a pull (yes, in this case, push == triggered pull,
>> which is safer), or using  other trigger forms (email, rss feed, ...).
>>
>> It's just something no one on the Fedora side has implemented.  I'm very
>> open to adding code to MM to assist with this, but I haven't ever gotten
>> around to it, and over the next few weeks I likely can't.  But if
>> someone wants to take a crack at it, I'd love the help!

   I don't have time to look at it now, but for what I remember the 
scripts should be quite easy to adapt to Fedora's needs.

> 
> Push mirroring is evil.

   Managing a tier-1 Debian mirror for years, I can't agree with that.

> If you can't make sure that the server has
> always enough resources for all projects to do a push mirroring
> simultaneously, then you get issues with traffic and high CPU loads.

   On my mirror, if the load is too high, my script would wait before 
rsyncing (and ignore new rsync push for the same project) until the load 
goes back to a more reasonnable level (and send me an email about it if 
the load stay high for too long, or is really too high).

   When pushing other mirror, if you push a high number of mirrors, it is 
quite easy not to send the PUSH signal to every mirrors at the same time, 
or to push let say 5 or 10 mirrors maximum at once if that is what your 
hardware/bandwidth allows you.

   And even if the tier-1 mirrors and the Fedora master would start to 
offer the possibility to get pushed, that won't necessarily force every 
mirrors to use it.  But a reliable tier-1 mirror that is pushed will 
attract many more mirrors because people will be confident that the data 
will be very up to date.

> Not to mention possible security implications - no matter how much you
> harden the ssh access and limit a key to a single command, it is still
> less safe than pure polling. Just consider the Debian openssh issue
> being detected in 2010 and some of us had ssh access with one of these
> keys.

   Well, even with that, it couldn't compromise a mirror that properly 
tied the ssh key with a specific command as it should when using the push 
mecanism (at most, it could create unnecessary load on the server, but not 
DoS it if the script is properly implemented).

   But as said, if you have the luxury of not needing a remote ssh access 
to your mirror, and are more worried about the bugs in SSH than the ones 
in rsync, your web server or ftp server, then you can simply not use the 
push mecanism at all.  But for the Debian and Ubuntu projects, it proved 
to be highly effective and very convenient.

   Have a nice day,

Simon Valiquette
http://gulus.USherbrooke.ca

--