[mirror-admin] fullfilelist (was Re: Please use --delay-updates)

[Josh Boyer]

On Wed, Apr 21, 2010 at 12:42:12PM -0400, Brian Long wrote:
>On 04/21/2010 11:39 AM, Matt Domsch wrote:
>> On Wed, Apr 21, 2010 at 09:38:44AM -0400, Brian Long wrote:
>>> I'm worried about the tiered mirror system and fullfilelist not being a
>>> good combination because a tier 2 or tier 3 might miss one or two of
>>> these files and be marked by MirrorManager as out-of-date.  If there was
>>> an email notification system inside MirrorManager which could alert
>>> mirror admins that their system fell out-of-date, this might help.
>> 
>> Would you prefer email per-host, or per-site (noting which host was
>> dropped) ?
>
>I would imagine per-site would be best.  I can always parse the email.
>
>I'd only like to see this if it's true that missing a single
>fullfilelist update necessitates the full disk scan of a normal rsync.

I'm confused on your concern here.  If the local mirror has a copy of the
fullfilelist file from a particular point in time, and fullfilelist is updated
on the master mirror X number of times after that, why does that matter?  The
local mirror would be doing a diff of their copy against whatever new copy they
pick up, which should contain all the changes to that file that have happened
since they originally got the fullfilelist.  Then you'd use that diff to sync
the things that changed.

Why does missing an update to fullfilelist necessitate a full disk scan?

josh

--