[mirror-admin] Crawling problem with my site

[James Hayden]

Strange. Thats what I did. I added the https URLs for each. I can the URLs listed for each mirror.

Example (cut and paste):

"Categories Carried

Hosts carry categories of software. Example Fedora categories include Fedora and Fedora Archive.

 [add]
Fedora Linux  
https://it-mirrors.evowise.com/fedora
Fedora EPEL  
https://it-mirrors.evowise.com/epel
"

Those craw date were cut and pasted from:
https://admin.fedoraproject.org/mirrormanager/site/mine

Hayden

----- Original Message -----
From: "Adrian Reber" <adrian at lisas.de>
To: mirror-list-d at redhat.com
Sent: Tuesday, July 19, 2016 10:28:35 AM
Subject: Re: Crawling problem with my site

On Tue, Jul 19, 2016 at 09:09:40AM -0700, Mike Iglesias wrote:
> I noticed yesterday that my site (ftp.uci.edu) has been marked as disabled
> because it couldn't be crawled.  When I click on the '[Log]' link next to when
> it was last crawled, I get a blank page, so I can't resolve whatever the
> problem is.

You have no URLs associated with your mirror. This is probably related
to the fact that all FTP URLs have been removed from the database and it
seems your mirror is only reachable via FTP.

> Are the date/times of the crawls GMT, EDT, ???

All times are in UTC.

> What are the IPs/address ranges of the crawlers?  This would help me track
> down what the issue is as I can look in logs to see what's going on.

If possible please add HTTP, HTTPS and RSYNC URLs to your mirror and
then you can re-enable it.

The log files are empty as they are rotated once a week and the old
crawler log files are not exported to web frontend systems.

		Adrian

--

--