[mirror-admin] push mirroring plans

[Matt_Domsch at Dell.com]

I'd like to have a session at the FUDCon F11 Hackfest / BarCamp event in
Boston in January to flesh out how to do "push mirroring" correctly.  If
you're thinking of attending FUDCon, let me know if you'd be interested
in helping think through this.

One big decision we need to make is who initiates the triggering.

In a traditional "pull mirror" system like we have, each mirror site
schedules an rsync job to happen at various times, and maybe there's new
content, maybe there's not.  Polling, over nearly 1TB of data.  Not so
nice.

In a Debian-style "push mirror", once the master mirrors have new
content, they trigger the Tier 1 mirrors to being to sync.  Once a Tier
1 has finished syncing, it triggers each of the Tier 2 mirrors
downstream of it.  This requires the downstream mirrors to coordinate
with their chosen upstream mirrors directly.

In a MirrorManager-like system, I think we can use the MM database, and
the fact all Tier 1 or "upstream" mirrors use the report_mirror script,
to have MM itself trigger the "downstream" mirrors when an "upstream"
mirror checks in.  We would need to include in the database the tree of
"upstream" vs "downstream" mirrors (there's a field for that already,
though it's not really used).  This would let the single point of
coordination continue to be the MM database as opposed to between every
set of mirror pairs.

I think we'd want to have a unique SSH keypair for each mirror stored in
the MM database, for the MM -> mirror trigger script.

Thoughts?

--
Matt Domsch
Linux Technology Strategist, Dell Office of the CTO
linux.dell.com & www.dell.com/linux


--