[mirror-admin] Gigabit NIC saturated.. [Re: Fedora 9 bitflip... slow becauseof the updates

[Brian Long]

On Wed, 2008-05-14 at 18:06 -0400, Paul Mezzanini wrote:
> On 5/14/08 4:19 PM, "Voll, Toivo" <toivo at acnet.usf.edu> wrote:
> 
> > 
> > Which leads me to two questions:
> > 1) The apache rewrite rule on the wiki page to prevent ranged http
> > requests produces a syntax error in my CentOS Apache install
> > (specifically the actual rule, the rest of the config is fine), any
> > ideas why it might fail or how to combat the download accelerators? (I
> > am already running connection limiting, but network installs get a bunch
> > of simultaneous requests going too, so I can't limit it too far.)
> > 
> 
> Is it just a generic "syntax error" or anything specific?  Can you attach
> the relevant part of your apache config file and the actual error output?
> 
> As for those pesky download accelerators...  Just today I blocked all
> resumes for .iso files on my server.  In roughly 15 seconds I had over 60
> hits for one file alone across 4 IP addresses.  I tried the block on the
> page in the past but it had very little effect.  After some snooping around,
> I learned that some flavor of accelerator just does not put an end on the
> range and dumps the connect after it gets the bytes it wants.  That bypasses
> the rules we have listed.
> 
> I looked into making every match give a 416 error because that looked to be
> the most relevant.  ( Requested Range Not Satisfiable )  The machines kept
> pounding on my server so I just set it to 403 before leaving.  They still
> kept pounding even with the 403 so I'm not totally sure how I will handle
> this yet.
> 
> How does everyone else handle this problem?

I maintain an internal mirror, so I don't personally have this issue,
but could you limit by User Agent or do download accelerators use common
user agents that you wouldn't want to block?

/Brian/

-- 
       Brian Long                             |       |
                                          . | | | . | | | .
                                              '       '
                                              C I S C O

--