[mirror-admin] master server sync stats and recommendations

[Axel Thimm]

Hi,

I think two items would improve the situation a lot:

a) Introduce a file checking method for mirrors to check whether a
   full rsync is needed (If it were just packages, the repomd file
   could be used).

   You could either have a dummy date file at the top of each module,
   or if you don't want to pollute the content with mirroring
   metadata, then use a separate module for all such rsync
   triggers. rsync mirrors could then make a short rsync on these date
   files and decide on rsyncing the content or not based on the files'
   changes (date and content, e.g. the files could be simply generated
   with something like 'TZ=UTC date > .fedoraupdates.lastchange')

   This would also help with staged tiering: For paranoic mirror
   managers (like myself), that are afraid of riding a stale mirror,
   the checking script could try both Red Hat's masters, as well as a
   tier 1/2 mirror (or more!) and pick the lower tier mirror if the
   dates are the same, otherwise it picks the master as a fallback. In
   general there could be multiple master mirrors in a preferred order
   and the mirror picks the one with the freshest content & higher
   preference.

   That way
   - mirror managers are quite confident that they have fresh data,
     e.g. as fresh as their last rsync launch was
   - tiered mirrors get used more
   - the content, especially security updates and rawhide propagates
     much faster
   - There are less full rsyncs to masters/tiered masters, since null
     rsyncs are filtered out.
   - There is no worry as to how often a mirror tries to rsync due to
     null rsyncs, since the only thing downloaded in that case would
     be the date files, which all toghether would be less than 1KB.

b) Manage tiering in Mirrormanager itself. The mirrors that want to
   attach to a master mirror apply through the web interface and
   either the master's mirror admin has given green light to all, or
   it is queued for approval. Then Mirrormanager generates an ACL list
   per master mirror, which the master mirror can automatically just
   rsync in.

-- 
Axel.Thimm at ATrpms.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
Url : http://mail.ale.org/pipermail/mirror-admin/attachments/20090422/443baf08/attachment.bin 
-------------- next part --------------
--