[mirror-admin] push mirroring plans

[Ken Tossell]

On Dec 16, 2008, at 11:17 PM, <Matt_Domsch at Dell.com> wrote:

> I think we'd want to have a unique SSH keypair for each mirror  
> stored in
> the MM database, for the MM -> mirror trigger script.

I understand SSH is used successfully for initiating syncs in a few  
big mirror hierarchies, but couldn't we use something less invasive?

I'd be interested in participating in a push mirroring system, but I  
definitely couldn't have any automatic logins. (We're using a  
distributed account system that's pretty strictly one login per  
person, and an account gets you into lots of different machines.) I'm  
sure there are a lot of others who couldn't allow SSH pushes, for  
similar or other reasons.

What would you all think of a system that works like this:
  - On master updates, the master server sends an HTTP query (like  
report_mirror) to admin.fp.org, telling MM to set all the tier-1  
mirrors' "needs to pull" flags.
  - Every k minutes (where k is something small, like 5-10), each  
tier-1 mirror queries MM, and if its flag is set, it syncs with the  
master. Once it's synced, it tells MM, which resets the tier-1 flag  
and sets the flags for that mirror's tier-2 clients.
  - Every 3k minutes, the tier-2 mirrors follow the same process.

Small updates, then, should propagate throughout tier 2 in less than  
an hour, without MM or upstream mirrors having any access to  
downstream hosts. The "Do I need to pull?" queries should be really  
easy to handle, as they wouldn't require any (expensive) database  
access.

- Ken
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.ale.org/pipermail/mirror-admin/attachments/20081217/80a1dc5e/attachment.html 
-------------- next part --------------
--