[mirror-admin] Breaking the 1TB limit?

[Mirrors Admin]

On Mon, Feb 22, 2010 at 01:08:36PM -0600, Matt Domsch wrote:
> Since the dawn of Fedora, we have, through informal agreement, limited
> the amount of content that Fedora provides for mirrors to carry, to
> under 1TB.  As obsolete releases expire, they get removed (more or
> less), and as new releases are available, they get added, yet we've
> managed to stay below 1TB (though sometimes dangerously close to it).
> 
> For some content, such as spins, videos, and the like, we have set up
> a second set of content, that is not widely mirrored, but is available
> should you wish to mirror it.  This is the "alt" content on alt.fp.o.
> Likewise, as we migrate obsolete releases from the master mirror, it's
> moved to archive.fp.o, and some mirrors also carry that for us.
> 
> We have over 9000 packages in the Fedora 13 repository, growing at
> over 1000/year.  We have quite a few Fedora Spins that would
> appreciate more widespread mirroring of the work they do.
> 
> My question for the mirrors: at what point can we increase beyond 1TB?
> Is that a meaningful number anymore?
> 
> In my mind, the question is two-fold:
> 
> 1) do many mirrors have more than 1TB they would be willing to use to
> host Fedora content?  If so, how many, and how much more?

For Redundancy I wouldn't mind it split up, so I could have fedora base in
location X while having the spins, videos and whatnot in location Y.  This
will give me the flexibility to spread it across my mirroring infrastructure
and use my disk space better. Busy stuff can go on beefier hardware, and not
as busy stuff can still be mirrored but somewhere that won't get hammered as
much.  (Or vice versa stuff not accessed as much can be "down" or slow on big
release days for Firefox, Ubuntu or other project I host)
> 
> 2) do we have the bandwidth, collectively, to mirror that much more
> content?  Due to our use of Tiering, we have done a decent job of
> getting content out to our mirrors faster than in the past.  If we are
> pushing that much more content, can we handle it?

bandwidth is not a problem. 

> 
> I welcome your feedback on this.  There is no formal decision being
> made, I'm just trying to gauge the abilities of our fantastic mirrors,
> without whom we'd be sunk.
> 
> Thanks,
> Matt
> Fedora Mirror Wrangler
> 
> --

--