[mirror-admin] where are all the indices of the repository?

[J.H.]

Paulo Licio de Geus wrote:
> J.H. wrote:
> 
>> Paulo Licio de Geus wrote:
>>> J.H. wrote:
>>>
>>>> Carlos Carvalho wrote:
>>>>>  >Why don't you want to use --delay-updates?
>>>>>
>>>>> Because of the disk hit. Fedora updates very often involve more than
>>>>> 10,000 files, and all these renames in sequence hit the disk hard. A
>>>>> few days ago an update of about 12,700 files took about 20min of
>>>>> renaming, and another a few days earlier of >20,000 took more than
>>>>> 33min. During these periods the number of transactions in the disks
>>>>> was
>>>>> around 98% of the maximum. Distributing the renames during the much
>>>>> longer download time avoids these peaks.
>>> Have you tried ionice -c3 -p <rsync_pid>?
>>>
>> That makes the assumption my I/O would ever be remotely nearly 'idle',
>> I've got 16 different things (distros mostly) on those disk, and I
>> have a tendency to move 200 - 300mbps constant off those machines
>> (with peaks up into the 1gbps range).  My I/O is never really 'idle',
>> so I would fall hopelessly behind on syncing.
> 
> Having "idle" (-c3) time means, I believe, when you have utilization
> below 100% (sar -d -p 3 should give a pretty good idea of the duty cycle
> on the disks). So if the primary service is "serving clients", then
> updating the mirror contents is not the main priority and could be run
> "off-line". We have all discussed ways of making sure the contents is
> never incoherent, despite the timing implied by each activity, so it
> doesn't matter if the update takes one hour or one and a half hours.
> 
> Now, how good the I/O scheduling algorithm is implemented in the Linux
> kernel is a differente story altogether, but I've played I little on my
> personal machine and my impression is that it mostly works. Why not give
> it a try on a server?

Really, my argument here isn't that idle there is an issue, it's that my 
disks are *CONSTANTLY* getting hammered, there really isn't much of 
anything I can do about this that would be sane.  Your suggesting that 
an update like that might take an hour or two (when set to idle like 
that) would be fine, but my disk is never idle like that, in fact it's 
*SLOW* right now with some of my machines with loads of 5 - 10, all of 
which is stalled on disk i/o.  During a Fedora release (for instance) 
loads have been seen to be well over 100 (I think the most I've 
personally seen lately is in the 500-600 in the last year or two) all of 
which is disk i/o bound, not cpu or memory.

- John 'Warthog9' Hawley

--