[mirror-admin] Mirrormanager failing

[Kevin Fenzi]

On Tue, 24 Apr 2012 11:00:54 -0500
Chris Adams <cmadams at hiwaay.net> wrote:

> I just replied to somebody's post on fedora-devel about this (and I'm
> not sure of the best place to send this): mirrormanger is not serving
> URLs:
> 
> $ wget -q -O-
> 'https://mirrors.fedoraproject.org/metalink?repo=fedora-16&arch=x86_64'
> <?xml version="1.0" encoding="utf-8"?> <metalink version="3.0"
> xmlns="http://www.metalinker.org/" type="dynamic" pubdate="Tue, 24
> Apr 2012 15:59:16 GMT" generator="mirrormanager"
> xmlns:mm0="http://fedorahosted.org/mirrormanager"> <!-- # Bad Request
> --> </metalink>
> $ 
> 
> I'm getting replies like that for Fedora (releases, updates, rawhide)
> and EPEL.  Looking at my monitoring, this started about 10am CDT (an
> hour ago as I send this).

Everything should be back now. 

It was some issue with entries in the db having a null value for
bandwidth, causing a cron job that updates mirror lists to break. 

We are: 

a) adding error checking to the cron job to avoid this breaking it and
spit out errors. 

b) asking mirrormanager developers to make sure that field cannot be
null. 

c) adding/adjusting docs we have internally on how to troubleshoot
this. 

d) adjusting our monitoring. 

> Just curious: what kind of monitoring is done for things like this in
> the Fedora infrastructure?

We have nagios checks for the metalinks, but sadly, the error was
internally only a 400 error, it was wrapped in a 200/OK, so the
monitoring was showing it up. We need to disect the result and make
sure it's returning valid mirrors. 

kevin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
Url : http://mail.ale.org/pipermail/mirror-admin/attachments/20120424/d2b33f80/attachment.bin 
-------------- next part --------------
--