[mirror-admin] Fwd: huge cleanups

[Stephen John Smoogen]

On 17 May 2016 at 22:05, Carlos <carlos at inf.ufpr.br> wrote:
> Yesterday the repository had about 2.6 million objects. Today it has about 1.5.
> The size also dropped significantly, though not as much. That's quite a
> cleanup, unusual for distros.
>
> I'm not complaining, just noticing in case there's something amiss.
>
> --

Thank you for catching that. I thought that Dennis sent this to the
mirror list but it was to a different list. I also sent my other
deletion to the wrong list also. I will send out an official notice.

1. Fedora Atomic has been moved out of the /pub/fedora/linux domain
and is now going through the koji squid proxies. The reason for this
was multiple:
  a. the ostree tool isn't aware of mirrormanager and the work to do
so was taking longer than expected so mirrors carrying this were not
seeing usage from people unless they hard coded to use a particular
mirror.
  b. ostree is a git based file system. Thus the number of objects in
it grows as more and more packages are updated. This has bonuses for
the devop who wants to rollback/forward various packages but it also
is very rsync bad as every file in the tree has to be stat() when the
demon checks what files have been updated. It took time to go over
900,000 files which took up 95% of a rsync of the fedora-enchilada.
That should be most of the 1.1 million objects you saw removed. [The
overall size of this wasn't too much about 250 GB for 21->rawhide
trees]

2. Fedora 21 was archived and removed from /pub/fedora/linux. These
files had been hardlinked with the /pub/archive/fedora/linux/ for the
last 3 weeks so when I removed them from fedora-enchilada there would
be drop in number of objects but not much in size.

-- 
Stephen J Smoogen.

--