[mirror-admin] New Mirror not being spidered and not showing up in mirror list

[Matt Domsch]

On Mon, Dec 14, 2009 at 02:51:07PM +1300, Dan Hawke wrote:
> I registered our mirror officially a few weeks ago in the hope that we could
> get it published to your mirrors page, and I have it happily checking in
> with report_mirrors showing plenty of directories...
> 
> However, it isn't showing up in any of the mirror finder options - not on
> the web page, not on the mirror locater script...  I suspect this might be
> because it's never been spidered.  However, I can't figure why it hasn't
> been spidered yet.
> 
> Our mirror is ucmirror.canterbury.ac.nz - for instance I see this in our
> mirrormanager screen:
> 
> Last Checked In: 2009-12-14 01:20:18
> Last Crawled:
> 
> Why aren't we getting crawled?

Because your Site is marked private.  Nearly all private mirrors are
unreachable from the public internet, so the crawler doesn't try.  MM
assumes that you run report_mirror regularly as you do, and trusts
that.  Private mirrors also don't appear on the publiclist pages.

Do you intend to be a public mirror?  If so, clear the "private"
checkbox on your Site's page.

> It's picking up categories, and I've linked them to URLs:
> Hosts carry categories of software. Example Fedora categories include Fedora
> Core and Fedora Extras.

yep, these all look good.

Also, be aware that due to the Fedora Infrastructure physical move
from one data center to another over the last few days, the crawlers
are offline until a few more things are fixed.  Whee!

Thanks,
Matt

-- 
Matt Domsch
Technology Strategist, Dell Office of the CTO
linux.dell.com & www.dell.com/linux

--