[mirror-admin] Mirror comes and goes on the mirrorlist

[Chris Adams]

Once upon a time, Matt Domsch <Matt_Domsch at dell.com> said:
> I just ran the crawler against your host, and for some reason it
> stopped after scanning quite a few directories.  It made it as far as:
> 
> http://mirror.hiwaay.net/pub/fedora/linux/extras/6/x86_64/repoview/layout
> 
> but didn't complete.  I'll investigate further.

Thanks for looking at this.  Checking my log, it made some odd-looking
requests (all of which were 404) after hitting the above directory:

209.132.177.200 mirror.hiwaay.net - [17/Mar/2008:08:40:05 -0500] "HEAD http://mirror.hiwaay.net/pub/fedora/linux/extras/development/SRPMS/synce-serial-0.9.1-2.fc7.src.rpm HTTP/1.1" 404 0 "-" "mirrormanager-crawler/0.1 (+http://hosted.fedoraproject.org/projects/mirrormanager)"
209.132.177.200 mirror.hiwaay.net - [17/Mar/2008:08:40:05 -0500] "HEAD http://mirror.hiwaay.net/pub/fedora/linux/extras/development/i386/yum-versionlock-1.1.3-1.fc7.noarch.rpm HTTP/1.1" 404 0 "-" "mirrormanager-crawler/0.1 (+http://hosted.fedoraproject.org/projects/mirrormanager)"
209.132.177.200 mirror.hiwaay.net - [17/Mar/2008:08:40:05 -0500] "HEAD http://mirror.hiwaay.net/pub/fedora/linux/extras/development/ppc/yum-versionlock-1.1.3-1.fc7.noarch.rpm HTTP/1.1" 404 0 "-" "mirrormanager-crawler/0.1 (+http://hosted.fedoraproject.org/projects/mirrormanager)"
209.132.177.200 mirror.hiwaay.net - [17/Mar/2008:08:40:05 -0500] "HEAD http://mirror.hiwaay.net/pub/fedora/linux/extras/development/x86_64/yum-versionlock-1.1.3-1.fc7.noarch.rpm HTTP/1.1" 404 0 "-" "mirrormanager-crawler/0.1 (+http://hosted.fedoraproject.org/projects/mirrormanager)"
209.132.177.200 mirror.hiwaay.net - [17/Mar/2008:08:40:05 -0500] "HEAD http://mirror.hiwaay.net/pub/fedora/linux/extras/internet2/i2.c HTTP/1.1" 404 0 "-" "mirrormanager-crawler/0.1 (+http://hosted.fedoraproject.org/projects/mirrormanager)"
209.132.177.200 mirror.hiwaay.net - [17/Mar/2008:08:40:05 -0500] "HEAD http://mirror.hiwaay.net/pub/fedora/linux/extras/mirrorlists/publiclist/index.html HTTP/1.1" 404 0 "-" "mirrormanager-crawler/0.1 (+http://hosted.fedoraproject.org/projects/mirrormanager)"
209.132.177.200 mirror.hiwaay.net - [17/Mar/2008:08:40:05 -0500] "HEAD http://mirror.hiwaay.net/pub/fedora/linux/extras/mirrorlists/publiclist/EPEL/5/index.html HTTP/1.1" 404 0 "-" "mirrormanager-crawler/0.1 (+http://hosted.fedoraproject.org/projects/mirrormanager)"
209.132.177.200 mirror.hiwaay.net - [17/Mar/2008:08:40:05 -0500] "HEAD http://mirror.hiwaay.net/pub/fedora/linux/extras/mirrorlists/publiclist/Fedora/7/x86_64/index.html HTTP/1.1" 404 0 "-" "mirrormanager-crawler/0.1 (+http://hosted.fedoraproject.org/projects/mirrormanager)"
209.132.177.200 mirror.hiwaay.net - [17/Mar/2008:08:40:06 -0500] "HEAD http://mirror.hiwaay.net/pub/fedora/linux/extras/mirrorlists/publiclist/Fedora/9-Alpha/x86_64/index.html HTTP/1.1" 404 0 "-" "mirrormanager-crawler/0.1 (+http://hosted.fedoraproject.org/projects/mirrormanager)"
209.132.177.200 mirror.hiwaay.net - [17/Mar/2008:08:40:06 -0500] "HEAD http://mirror.hiwaay.net/pub/fedora/linux/extras/mirrorlists/static/css/fedora.css HTTP/1.1" 404 0 "-" "mirrormanager-crawler/0.1 (+http://hosted.fedoraproject.org/projects/mirrormanager)"
209.132.177.200 mirror.hiwaay.net - [17/Mar/2008:08:40:06 -0500] "HEAD http://mirror.hiwaay.net/pub/fedora/linux/extras/mirrorlists/static/images/fedora/images/line-bottom.png HTTP/1.1" 404 0 "-" "mirrormanager-crawler/0.1 (+http://hosted.fedoraproject.org/projects/mirrormanager)"
209.132.177.200 mirror.hiwaay.net - [17/Mar/2008:08:40:06 -0500] "HEAD http://mirror.hiwaay.net/pub/fedora/linux/extras/mirrors/samples/custom-xmlrpc-calls.py HTTP/1.1" 404 0 "-" "mirrormanager-crawler/0.1 (+http://hosted.fedoraproject.org/projects/mirrormanager)"
209.132.177.200 mirror.hiwaay.net - [17/Mar/2008:08:40:06 -0500] "HEAD http://mirror.hiwaay.net/pub/fedora/linux/extras/mirrors/sqlobject-history/2006-11-24/Config_postgres.sql HTTP/1.1" 404 0 "-" "mirrormanager-crawler/0.1 (+http://hosted.fedoraproject.org/projects/mirrormanager)"
209.132.177.200 mirror.hiwaay.net - [17/Mar/2008:08:40:06 -0500] "HEAD http://mirror.hiwaay.net/pub/fedora/linux/extras/mirrors/static/images/fedora/images/line-bottom.png HTTP/1.1" 404 0 "-" "mirrormanager-crawler/0.1 (+http://hosted.fedoraproject.org/projects/mirrormanager)"
209.132.177.200 mirror.hiwaay.net - [17/Mar/2008:08:40:06 -0500] "HEAD http://mirror.hiwaay.net/pub/fedora/linux/extras/server-scripts/test/server_tester.py HTTP/1.1" 404 0 "-" "mirrormanager-crawler/0.1 (+http://hosted.fedoraproject.org/projects/mirrormanager)"


-- 
Chris Adams <cmadams at hiwaay.net>
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.

--