[mirror-admin] Slow syncing?

[J.H.]

On 01/06/2010 10:24 AM, Chris Adams wrote:
> Once upon a time, Marek Mahut <mmahut at redhat.com> said:
>> Hello Chris,
>>
>> Chris Adams wrote:
>>> Once upon a time, Marek Mahut <mmahut at redhat.com> said:
>>>> Hello everyone,
>>>>
>>>> We need more details on this issue. Can you please report what's your IP
>>>> address, time of sync, slow speed and also what speed did you get before
>>>> migration?
>>>>
>>>> Thank you for your help and data,
>>>
>>> Host: mirror.hiwaay.net [216.180.99.217]
>>> Time: whenever fullfilelist is updated (speeds seem consistent all day)
>>> Speed: around 2 megabits per second (IIRC it was at least 10 Mbps before)
>>
>> Do you get this speed only for fullfilelist update?
>>
>> What is the speed of actual package transfer?
> 
> That is the total download speed.
> 
>> Can you try running this rsync command in an empty directory?
>>
>> rsync -av --progress --verbose rsync://download.fedora.redhat.com/fedora-epel .
> 
> opening tcp connection to download.fedora.redhat.com port 873
> sending daemon args: --server --sender -vvlogDtpr . fedora-epel/ 
> ----------------------------
> Fedora Master Mirror Servers
> ----------------------------
> Modules for Fedora Core and Extras have been removed, as this content is
> no longer updated. See the instructions below for how to mirror current
> content.
> 
> See http://fedoraproject.org/wiki/Infrastructure/Mirroring for
> instructions.
> ----------------------------
> 
> receiving file list ... 
> [receiver] expand file_list pointer array to 262144 bytes, did move
> [receiver] expand file_list pointer array to 524288 bytes, did move
> 66957 files to consider
> delta-transmission enabled
> ./
> 4AS -> 4
> 4ES -> 4
> 4WS -> 4
> 5Client -> 5
> 5Server -> 5
> RPM-GPG-KEY-EPEL
>         1698 100%    1.62MB/s    0:00:00 (xfer#1, to-check=66950/66957)
> fullfilelist
>      2965247 100%  181.65kB/s    0:00:15 (xfer#2, to-check=66949/66957)
> 4/
> 4/SRPMS/
> 4/SRPMS/Ajaxterm-0.10-5.el4.src.rpm
>        39079 100%   34.20kB/s    0:00:01 (xfer#3, to-check=66946/66957)
> 4/SRPMS/BibTool-2.48-6.el4.src.rpm
>       639889 100%  382.66kB/s    0:00:01 (xfer#4, to-check=66945/66957)
> 4/SRPMS/CGSI-gSOAP-1.3.3.2-2.20090920cvs.el4.src.rpm
>        57415 100%   86.00kB/s    0:00:00 (xfer#5, to-check=66944/66957)
> 4/SRPMS/DMitry-1.3a-2.el4.src.rpm
>       133651 100%   82.61kB/s    0:00:01 (xfer#6, to-check=66943/66957)
> 4/SRPMS/Django-1.1.1-1.el4.src.rpm
>      5608901 100%  234.47kB/s    0:00:23 (xfer#7, to-check=66942/66957)
> 4/SRPMS/GMT-4.3.1-2.el4.src.rpm
>      4772566 100%  173.73kB/s    0:00:26 (xfer#8, to-check=66941/66957)
> 4/SRPMS/GMT-coastlines-1.10-2.src.rpm
>     42538589 100%  164.76kB/s    0:04:12 (xfer#9, to-check=66940/66957)
> 4/SRPMS/GMT-doc-4.3.1-3.src.rpm
> 
> 

Just so everyone knows, I'm running a parse on our sync logs, which
thankfully go back to ~2007.  I should have a pretty graph in a little
bit about the number of files, speed of transfer, etc for all of 2009 in
a little bit.

- John 'Warthog9' Hawley

--