[mirror-admin] Blocking ips on dl.fedoraproject.org (Or Please update your mirrors in mirror-manager)

[Carlos]

Stephen John Smoogen (smooge at gmail.com) wrote on Mon, Mar 28, 2016 at 03:59:25PM BRT:
> The server we are seeing is
> 
> download05/rsyncd-fedora.log:2016/03/28 17:20:05 [34236] connect from
> sagres.c3sl.ufpr.br (200.236.31.1)
> download05/rsyncd-fedora.log:2016/03/28 17:20:05 [34236] rsync on
> fedora-enchilada0/fullfilelist from sagres.c3sl.ufpr.br (200.236.31.1)
> 
> Not
> 
> [smooge at smooge-laptop00 Rsync-Stats]$ host fedora.c3sl.ufpr.br
> fedora.c3sl.ufpr.br has address 200.236.31.8
> fedora.c3sl.ufpr.br has IPv6 address 2801:82:80ff:8000::9

Mirrormanager asks for the site name and the IPs of the rsync servers. This
doesn't mean they should be the same machine. In our case it is, but it has
several IPs. All you should check is if the IPs used are those in mirrormanager
for the site.

> >> Using the last-sync to schedule updates when they actually occur can
> >> help lower rsync usage.
> >
> > Certainly. Here's the log from one of these runs:

> > Timestamp upstream nao e' mais recente
> > Abortando
> >
> > The last sentences mean "Timestamp upstream is not more recent [then local
> > one]. Aborting"
> >
> 
> What was the command you are using to generate that one? The reason is
> I am trying to see if it generates a large load on the backend NFS and

Obviously it doesn't generate a large load, otherwise I wouldn't use it...

We're not the bad guys you're looking for. As I already posted in this list
years ago, I use fullfilelist:

rsync dl.fedoraproject.org/fullfilelist > time-stamp

So rsync just does a "ls -l" of the file.

> if there is a better way for us to get that information to you.

Sure there is, as I already said here several times: do a decent fullfilelist
instead of what you have. It should include the timestamps and sizes, done with 

TZ=UTC rsync --no-h -r /path/to/repo > good-fullfilelist

Then C3SL will not scan you at all; we just download good-fullfilelist (if it's
time-stamp shows it's changed since our last update) and compare with ours.
Then we directly download only what we need. Zero devastating disk scans, both
here and at the master.

Other mirrors won't use it but if a single mirror, which updates very often,
does it's already worth it.

--