[mirror-admin] dl.fedoraproject.org - max connections reached

[Chuck Anderson]

It looks like q-f-m is in a loop of retries.  The first two tries got
most of the files (754GB and 14GB) with respectable transfer rates
(7.8MB/sec and 2.6MB/sec).  Then things went south.  I'm not sure if I
should kill q-f-m and restart it or what.

>> Calling /usr/bin/rsync -aSH -f R .~tmp~ --stats --delay-updates --out-format=@ %i  %n%L -v -v --files-from=master-transferlist.sorted rsync://dl05.fedoraproject.org/fedora-buffet0/ /srv/ftp/pub/fedoraproject.org
sent 900595689 bytes  received 753237293272 bytes  7830276.96 bytes/sec

>> Calling /usr/bin/rsync -aSH -f R .~tmp~ --stats --delay-updates --out-format=@ %i  %n%L -v -v --files-from=master-transferlist.sorted rsync://dl05.fedoraproject.org/fedora-buffet0/ /srv/ftp/pub/fedoraproject.org
sent 14450699 bytes  received 14232793225 bytes  2599150.58 bytes/sec

>> Calling /usr/bin/rsync -aSH -f R .~tmp~ --stats --delay-updates --out-format=@ %i  %n%L -v -v --files-from=master-transferlist.sorted rsync://dl05.fedoraproject.org/fedora-buffet0/ /srv/ftp/pub/fedoraproject.org
sent 2869733 bytes  received 41227635 bytes  12907.18 bytes/sec

>> Calling /usr/bin/rsync -aSH -f R .~tmp~ --stats --delay-updates --out-format=@ %i  %n%L -v -v --files-from=master-transferlist.sorted rsync://dl05.fedoraproject.org/fedora-buffet0/ /srv/ftp/pub/fedoraproject.org
sent 2869733 bytes  received 41227635 bytes  5020.48 bytes/sec

>> Calling /usr/bin/rsync -aSH -f R .~tmp~ --stats --delay-updates --out-format=@ %i  %n%L -v -v --files-from=master-transferlist.sorted rsync://dl05.fedoraproject.org/fedora-buffet0/ /srv/ftp/pub/fedoraproject.org
sent 2869733 bytes  received 41227635 bytes  4047.30 bytes/sec

>> Calling /usr/bin/rsync -aSH -f R .~tmp~ --stats --delay-updates --out-format=@ %i  %n%L -v -v --files-from=master-transferlist.sorted rsync://dl05.fedoraproject.org/fedora-buffet0/ /srv/ftp/pub/fedoraproject.org
sent 2869733 bytes  received 41227635 bytes  7598.41 bytes/sec

>> Calling /usr/bin/rsync -aSH -f R .~tmp~ --stats --delay-updates --out-format=@ %i  %n%L -v -v --files-from=master-transferlist.sorted rsync://dl05.fedoraproject.org/fedora-buffet0/ /srv/ftp/pub/fedoraproject.org
sent 2869733 bytes  received 41235139 bytes  11356.99 bytes/sec

>> Calling /usr/bin/rsync -aSH -f R .~tmp~ --stats --delay-updates --out-format=@ %i  %n%L -v -v --files-from=master-transferlist.sorted rsync://dl05.fedoraproject.org/fedora-buffet0/ /srv/ftp/pub/fedoraproject.org
sent 3329550 bytes  received 54057222 bytes  17964.24 bytes/sec

On Wed, Mar 08, 2017 at 09:03:41AM -0500, Chuck Anderson wrote:
> Due to the Fedora 26 branching last week, many mirrors are having
> trouble completing a sync.  It was taking my mirror over 24 hours to
> even get a file list before file transfers could begin , so I had to
> give up with the regular rsync method and use quick-fedora-mirror.
> That is working now, but even so it is still taking over 24 hours to
> complete the file transfers.  I suspect that the trees are not
> hardlinked completely and it is downloading way more than necessary.
> 
> On Wed, Mar 08, 2017 at 11:01:46AM +0000, HEAnet Mirror Admin wrote:
> > Hi,
> > 
> > We've had trouble connecting to dl.fedoraproject.org for the last few
> > days - it appears to be hitting the maximum number of rsync
> > connections quite frequently.
> > 
> > rg

--