[mirror-admin] Blocking ips on dl.fedoraproject.org (Or Please update your mirrors in mirror-manager)

[Stephen John Smoogen]

On 28 March 2016 at 16:46, Carlos <carlos at inf.ufpr.br> wrote:
> Stephen John Smoogen (smooge at gmail.com) wrote on Mon, Mar 28, 2016 at 03:59:25PM BRT:
>> The server we are seeing is
>>
>> download05/rsyncd-fedora.log:2016/03/28 17:20:05 [34236] connect from
>> sagres.c3sl.ufpr.br (200.236.31.1)
>> download05/rsyncd-fedora.log:2016/03/28 17:20:05 [34236] rsync on
>> fedora-enchilada0/fullfilelist from sagres.c3sl.ufpr.br (200.236.31.1)
>>
>> Not
>>
>> [smooge at smooge-laptop00 Rsync-Stats]$ host fedora.c3sl.ufpr.br
>> fedora.c3sl.ufpr.br has address 200.236.31.8
>> fedora.c3sl.ufpr.br has IPv6 address 2801:82:80ff:8000::9
>
> Mirrormanager asks for the site name and the IPs of the rsync servers. This
> doesn't mean they should be the same machine. In our case it is, but it has
> several IPs. All you should check is if the IPs used are those in mirrormanager
> for the site.
>

But I am not seeing the ip listed in mirrormanager.

OK that is a failure on our part. I am going to need the ips of the
rsync servers if I am going to put in firewall rules for tier0 and
tier1 mirrors. I will need all of the ips that could be used or we
will run into this again. I will have to figure out how to do this
later.

>> >> Using the last-sync to schedule updates when they actually occur can
>> >> help lower rsync usage.
>> >
>> > Certainly. Here's the log from one of these runs:
>
>> > Timestamp upstream nao e' mais recente
>> > Abortando
>> >
>> > The last sentences mean "Timestamp upstream is not more recent [then local
>> > one]. Aborting"
>> >
>>
>> What was the command you are using to generate that one? The reason is
>> I am trying to see if it generates a large load on the backend NFS and
>
> Obviously it doesn't generate a large load, otherwise I wouldn't use it...
>
> We're not the bad guys you're looking for. As I already posted in this list
> years ago, I use fullfilelist:
>
> rsync dl.fedoraproject.org/fullfilelist > time-stamp
>
> So rsync just does a "ls -l" of the file.
>
>> if there is a better way for us to get that information to you.
>
> Sure there is, as I already said here several times: do a decent fullfilelist
> instead of what you have. It should include the timestamps and sizes, done with
>
> TZ=UTC rsync --no-h -r /path/to/repo > good-fullfilelist
>
> Then C3SL will not scan you at all; we just download good-fullfilelist (if it's
> time-stamp shows it's changed since our last update) and compare with ours.
> Then we directly download only what we need. Zero devastating disk scans, both
> here and at the master.
>

Thank you. While you may have posted this in the past.. I had
forgotten it. I have now saved this for future implementation.

> Other mirrors won't use it but if a single mirror, which updates very often,
> does it's already worth it.
>
> --



-- 
Stephen J Smoogen.

--