[mirror-admin] mirroring containers?

[Stephen John Smoogen]

On 7 November 2016 at 15:35, Matthew Miller <mattdm at mattdm.org> wrote:
> On Mon, Nov 07, 2016 at 11:29:32AM -0500, Stephen John Smoogen wrote:
>> I think it is hard to answer that question without a couple of things:
>> 1) How much disk space are you looking at?
>
> As a rough estimate, less than a gigabyte to start with, but growing
> fairly rapidly. I think "as big as traditional Fedora is now in three
> years" is probably realistic -- for the optimistic/successful end of
> the spectrum.
>

So from 1 GB to 1 TB in 3 years?


>> 2) How many files in that disk space are you looking at?
>
> There are, excitingly, two opposite answers for this. For OCI/Docker
> containers, an order of magnitude or so smaller than an RPM tree, since
> each image is an aggregate. For ostree, though, it's currently a lot of
> small files, since each RPM is exploded and tracked separately.
>

Yeah.. the OStree doesn't work for mirrors. It was causing all kinds
of problems with mirrors getting behind or constantly stale and the
Fedora mirrors causing problems with NFS. A completely different way
to store and mirror those files is needed at both ends.

>> 3) How much churn and what kind of churn is it going to be? If the
>> file names stay the same but the contents change then it could be
>> possible to save via rsync delta patching but if it is completely
>> different files then its pure downloads each time.
>
> The goal is to have less churn than we have with Fedora updates now;
> except for critical/important security updates, each container will be
> released in an updated batch every two weeks (assuming that any updates
> even apply to that container). We may also separate out "stable",
> "devel", and "rawhide/nightly" — and we could do that in a
> mirror-friendly way.
>

Does ostree change daily with updates?

>> 4) How do mirrors advertise this to customers?
>
> Two ways. First, system tools would use mirrormanager in a way similar
> to what yum/dnf do now. Second, we *probably* will have a web-based
> interface of some sort, although that's TBD.
>
> --
> Matthew Miller
> <mattdm at fedoraproject.org>
> Fedora Project Leader
>
> --



-- 
Stephen J Smoogen.

--