[mirror-admin] Breaking the 1TB limit?

[Stephen John Smoogen]

On Tue, Feb 23, 2010 at 1:05 PM, J.H. <warthog9 at kernel.org> wrote:
...
>> 2) do we have the bandwidth, collectively, to mirror that much more
>> content?  Due to our use of Tiering, we have done a decent job of
>> getting content out to our mirrors faster than in the past.  If we are
>> pushing that much more content, can we handle it?
>
> Since we have tiering we have solved a *LOT* of our problems with
> bandwidth, but your right that much more content means that much more
> bandwidth load on the mirrors.  Personally I think we can handle it
> collectively, assuming your not expecting 500G churn on a daily basis.
>
> It might help us better understand the issue if you could give us a
> better idea of what additional content we are talking about?  Is it just
> the growth of packages?  Is there other content that will cause this
> increase?

My expectation is that we are looking at regular growth and increased
'wants' of some contributers. The spins would probably increase disk
space usage by 10 GB per release, and we would see the usual 10%
growth of packages per release. I would say we are wanting to move our
promised limit to 1.2 TB for the entirety of Fedora active for the
next 2 years with archived data being a lot more.

I would hope that we don't reach 500GB daily churn til 2020 or so. But
128 bit computers should be standard by then with 10 Tbit the standard
network link.

>> I welcome your feedback on this.  There is no formal decision being
>> made, I'm just trying to gauge the abilities of our fantastic mirrors,
>> without whom we'd be sunk.
>
> It takes a good wrangler to herd all these cats ;-)

Matt and the rest of the mirror team have done a much better job than
I ever did... with a lot more patience too.

> - John 'Warthog9' Hawley
>
> --
>



-- 
Stephen J Smoogen.

Ah, but a man's reach should exceed his grasp. Or what's a heaven for?
-- Robert Browning

--