[mirror-admin] Your bandwidth # in Fedora MirrorManager is meaningful now

[Matt Domsch]

On Sun, Jun 29, 2008 at 11:10:08PM -0500, Matt Domsch wrote:
> Tonight I deployed a slight update to MirrorManager that I'd been
> wanting to do for a while.  The basic selection algorithm for choosing
> the order in which to return mirrors to clients remains the same:
> prefer same netblocks, internet2 in same country if on internet2, same
> country, same continent, then global, in that order.  However, until
> now, faster mirrors (those with larger bandwidths) were returned with
> equal probability as slower mirrors.  This means not enough traffic
> was being sent to our gigabit-connected mirrors, and too much traffic
> was being sent to our slower mirrors.
> 
> The "Bandwidth_int" value is now used as a weight, and instead of
> using python's random.shuffle() routine, I wrote a weighted_shuffle()
> that behaves more like a raffle.  Mirrors with larger values of
> 'bandwidth_int' get more raffle tickets, thus a higher probability of
> being returned earlier in the mirrorlist selection.  Slower mirrors
> can still be returned as the first mirrorlist selection because this
> is based on probabilities, but should not be overwhelmed anymore.

One slight change I made today, at the suggestion of Ken Tossell.  I
reverted to using random.shuffle() for the netblock and internet2
sublists, and bumped use of the internet2 lists up ahead of the
'specified country' list.

The 'Bandwidth_int' value represents the bandwidth available to
generic non-local users.  Mirrors serving local (same netblock)
users or Internet2 users often have significantly more bandwidth
available for those users.  Thus there's no win in using the
'Bandwidth_int' as a weight - it's meaninglss in those contexts.

Also, it was a bug that if users specified a country in their URLs
(e.g. appended &country=us), they would get sent to non-Internet2
mirrors in that country before getting sent to Internet2 mirrors in
that country if they could.  I fixed this bug.

Let me know how this goes for you, and we can adjust over time.  If
the weighting system is too skewed so larger bandwidth mirrors are
getting swamped, we can look at using log2(bandwidth) for the weight
instead of simply bandwidth.

Thanks,
Matt

-- 
Matt Domsch
Linux Technology Strategist, Dell Office of the CTO
linux.dell.com & www.dell.com/linux

--