[mirror-admin] Fedora mirror disk usage

[Dennis Gilmore]

If you do not follow hardlinking you will probably get what you are seeing.  
we make extensive use of hardlinks with duplicated content all over /pub  if 
you add -H to your rsync commands you should see a significant reduction in 
disk use.

you should only be seeing 
240G	/pub/fedora/linux/development/rawhide/
256G	/pub/fedora/linux/development/24

I checked thse values on the master mirror

If you are hardlinking and seeing more than that then check for stray data in 
your tree. If you want to share how you are mirroring content, we can try help 
you optimise it.

Dennis

On Monday, March 21, 2016 9:07:17 AM CDT Stephen John Smoogen wrote:
> On 20 March 2016 at 06:48, Ján ONDREJ (SAL) <ondrejj at salstar.sk> wrote:
> > Hello,
> > 
> >   looks like Fedora's download content is daily increasing.
> > 
> > I removed some non Fedora data from my mirror, but looks like it's still
> > not enough. Currently development content is more than 1 TB of data.
> > Is this normal? How many disk space is required to mirror Fedora?
> 
> Also check to see if you have a bunch of directories like /.~tmp~/
> filled with files. I have seen this with a couple of bad rsync where
> the delayed updates is not cleaning itself up for some reason and you
> have gigabytes of trash files that are still being used.
> 
> > [root at ftp]# du -sh /home/ftp/pub/fedora/linux/development/*
> > 675G    /home/ftp/pub/fedora/linux/development/rawhide
> > 542G    /home/ftp/pub/fedora/linux/development/24
> > 
> > This graph shows, how many data was added to our mirror storage:
> > 
> > http://www.salstar.sk/trafgrapher/nagios-service.html?n=/temp/perf/ftp;dis
> > k_usage;ftp_nrpe_disk_opt__opt&i=8766h&u=B
> > 
> > You can see, that last month increased at least 1 TB of data (I think
> > aprox
> > 1.5 TB with removed data).
> > 
> > Mey be some cleaning scripts on Fedora inftrastructure are broken and
> > don't remove old content. Here are usages for current stable releases:
> > 
> > 202G    /home/ftp/pub/fedora/linux/releases/21
> > 218G    /home/ftp/pub/fedora/linux/releases/22
> > 230G    /home/ftp/pub/fedora/linux/releases/23
> > 
> >                                                         SAL
> > 
> > --



--