[ale] to speed up your internet connection, slow it down (buffer bloat)

Sat Jul 7 16:12:12 EDT 2012

Hey Ron,

If you like, I can lend my 20x3M comcast line to you for testing. I can
throw up an quick knoppix box and you can test away with your own shell.
Its not really being used (except for me checking emails).  Its a Walking
Dead marathon today, so I doubt I will get any real work done.

Hit me off list for IP and login details.

-Erik-

On Sat, Jul 7, 2012 at 2:56 PM, Ron Frazier (ALE) <
atllinuxenthinfo at techstarship.com> wrote:

> ** Hi Erik,
>
> I haven't had a chance to do any testing. I'm not sure if I even can. Here
> machine runs Windows and is a locked down corporate computer that I have
> little access to. I can't install anything without their permission. I know
> some of her coworkers have been complaining, so I know that the problem is
> often on their side of the fence. I just wanted to eliminate any possible
> hitches on my end. My routers are basic off the shelf Netgear boxes. They
> have very little in the way of configuration settings. My machines are
> sometimes running Windows and sometimes Linux, but they don't have any
> visibility into her data traffic.
>
> She runs a citrix remote desktop system. So, she is viewing the video
> output of a remote system at all times. Presumably, this generates a fair
> amount of continuous time sensitive data traffic. Every mouse click she
> makes, every move she makes in the program she's running is sent to the
> remote system and the results are sent back. If the connection is disrupted
> for more than a second or so, her system complains about it. At this point,
> I cannot tell if my changes have made any substantial difference. They were
> more of a preemptive change, to sort of head off any local problems before
> they happen. If I come up with any notable results, or any viable way to
> test things, I'll certainly be glad to share.
>
> Here's something you can do to test the latencies on your own network.
>
> There's a tool called Netalyzr that's put out by the International
> Computer Science Institute at Berkeley. You will have to trust Berkeley and
> turn on javascript for this to work. This performs VERY extensive testing
> on your network and reports some findings to their study.
>
> http://netalyzr.icsi.berkeley.edu/index.html
>
> Here's something else which you could try. Turn off all QOS or traffic
> shaping systems on your router closest to the internet, and on any other
> routers or switches between your PC and the internet. Try to upload a
> really big file, like multiple GB to dropbox or something. While that's
> uploading, try to download another large file at the same time, say an ISO
> or something. The upload can disrupt the download because the ACK packets
> going back upstream may have a hard time getting through. You could also
> try a VOIP call or some gaming, etc. while the upload is in progress. If
> the testing with large downloads, VOIP, or gaming fails, it's probably
> because you have some large buffers somewhere and because they're getting
> full. If those tests don't fail, you may not have large buffers, or there
> may be traffic shaping going on that you don't know about, maybe in the
> cable modem. Again, I don't know about all the nitty gritty details of
> TCP/IP throttling, so I'm not an expert on the topic.
>
> Good luck. Let us know if you find out anything new for your network.
>
> Sincerely,
>
> Ron
>
>
>
> --
>
> Sent from my Android Acer A500 tablet with bluetooth keyboard and K-9 Mail.
> Please excuse my potential brevity.
>
> (To whom it may concern. My email address has changed. Replying to former
> messages prior to 03/31/12 with my personal address will go to the wrong
> address. Please send all personal correspondence to the new address.)
>
> (PS - If you email me and don't get a quick response, you might want to
> call on the phone. I get about 300 emails per day from alternate energy
> mailing lists and such. I don't always see new email messages very
> quickly.)
>
> Ron Frazier
> 770-205-9422 (O) Leave a message.
> linuxdude AT techstarship.com
>
>
> Erik Mathis <erik at mathists.com> wrote:
>>
>> I was wondering if you had done any pre/post iperf (or something smiler)
>> testing? If so, can you share the results?
>>
>> -Erik-
>>
>> On Fri, Jul 6, 2012 at 5:13 PM, Ron Frazier (ALE) <
>> atllinuxenthinfo at techstarship.com> wrote:
>>
>>> Hi all,
>>>
>>> I want to share some information about a phenomenon which can
>>> dramatically slow down your internet connection, or connections within a
>>> LAN some times. It's called buffer bloat. I first heard about it over on
>>> the NTP questions list. I don't remember why that came up, probably related
>>> to network latencies for NTP servers. Then, later, Steve Gibson discussed
>>> it on the Security Now podcast. I've provided several links below for those
>>> who wish to research it.
>>>
>>> For those not familiar, buffer refers to a memory queue in a router or
>>> other networking gear. The problem occurs when you go from a large
>>> bandwidth pipe to a smaller bandwidth pipe, such as the transition from
>>> your LAN to the internet WAN. At this point, you might go from 100 Mbps or
>>> 1 Gbps bandwidth to something like 3 Mbps or 20 Mbps or 50 Mbps or
>>> whatever. The point is, that it is a dramatic reduction in bandwidth.
>>>
>>> So, if you're trying to transmit to the internet at 100 Mbps and it can
>>> only take 20 Mbps, the link will become saturated. Without buffers or
>>> queues, about 4/5 of the packets will be dropped. The system will rapidly
>>> recognize that it cannot go that fast and it will scale down to something
>>> which the link can support.
>>>
>>> However, with large buffers, which many routers have, the problem
>>> becomes much worse. Let's say the router has a 4 M Byte or approximately 40
>>> M bit ram buffer on it's outbound transmission channel. Your computer fills
>>> that buffer in about 4/10 sec, but, that buffer is going to take 2 sec to
>>> empty out sending the data to the internet. While I don't understand all
>>> the technical magic that happens, I do understand that the normal automatic
>>> throttling systems no longer work. So, your computer might be seeing a 2
>>> sec delay to get packets out on the internet while they meander through the
>>> buffer on a first in first out basis.
>>>
>>> There is a new intelligent packet dropping algorithm called CODEL that
>>> may be the solution. The bufferbloat site mentions it, and Steve did a
>>> podcast talking about it. It shows great promise, however, most routers
>>> don't implement the algorithm, and many probably never will get upgraded,
>>> including many home routers.
>>>
>>> So, here, as I understand it, is a way you can work around the problem.
>>>
>>> My wife works from home sometimes and uses a VPN back to work.
>>> Sometimes, here system locks up and says the connection is lost. I have as
>>> many as 7 devices sharing the same internet connection, so her system may
>>> be experiencing congestion. I suspect that many times, the problem is on
>>> the other end at her office, but just in case, I decided to tweak the
>>> router. I turned on a QOS (quality of service) setting and told it to
>>> prioritize her data traffic over mine. I also made some changes to avoid
>>> any possible buffer bloat problem.
>>>
>>> The buffer bloat problem only shows up when the buffer fills. By the
>>> way, a clogged upstream buffer can shut down downloads too, since, during
>>> downloads, all tcp packets have to be acknowledged, and those
>>> acknowledgements must go upstream. A clogged buffer can essentially make
>>> your Internet connection almost unusable. I think this is what happens at
>>> many coffee shops. If you can't run CODEL or something like it, one way to
>>> prevent the problem is to make sure the buffer never fills up. One way to
>>> do that is to limit your upstream bandwidth to something less than what
>>> it's possible to do. In my case, the QOS menu of the router allows me to
>>> limit upstream bandwidth. I used speedtest.net to test the system. I
>>> was able to get a peak upstream bandwidth of 5.6 Mbps. So, I set the QOS
>>> controls on the router to limit the upstream bandwidth to 5 Mbps.
>>> Theoretically, this should mean that the outbound buffer on my router never
>>> will fill up because it's always emptying out faster than I'm putting data
>>> in. Theoretically, that should prevent the buffer bloat problem on my LAN.
>>> This, combined with prioritization of my wife's data, will hopefully solve
>>> her data problems.
>>>
>>> If you've had experience with this problem, please share what you
>>> learned and what you did about it.
>>>
>>> If you need info on the down and dirty operation of TCP/IP, ask some of
>>> the other wizards on the list.
>>>
>>> Hope this is helpful.
>>>
>>> Sincerely,
>>>
>>> Ron
>>>
>>> links below
>>>
>>> ----------------------
>>>
>>>
>>> http://en.wikipedia.org/wiki/Buffer_bloat
>>>
>>> http://www.bufferbloat.net/
>>>
>>> Steve Gibson discusses buffer bloat on the Security Now podcast episode
>>> 345.
>>> He introduces a potential solution, CODEL, developed by industry
>>> researchers, in episode 359.
>>>
>>> http://www.grc.com/securitynow.htm - Reference episodes 345 and 359.
>>>
>>> http://twit.tv/show/security-now/345
>>> http://media.grc.com/sn/sn-345.mp3
>>>
>>> http://twit.tv/show/security-now/359
>>> http://media.grc.com/sn/sn-359.mp3
>>>
>>>
>>>
>>> --
>>>
>>> Sent from my Android Acer A500 tablet with bluetooth keyboard and K-9
>>> Mail.
>>> Please excuse my potential brevity.
>>>
>>> (To whom it may concern. My email address has changed. Replying to former
>>> messages prior to 03/31/12 with my personal address will go to the wrong
>>> address. Please send all personal correspondence to the new address.)
>>>
>>> (PS - If you email me and don't get a quick response, you might want to
>>> call on the phone. I get about 300 emails per day from alternate energy
>>> mailing lists and such. I don't always see new email messages very
>>> quickly.)
>>>
>>> Ron Frazier
>>> 770-205-9422 (O) Leave a message.
>>> linuxdude AT techstarship.com
>>>
>>> _______________________________________________
>>> Ale mailing list
>>> Ale at ale.org
>>> http://mail.ale.org/mailman/listinfo/ale
>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>> http://mail.ale.org/mailman/listinfo
>>>
>>>
>>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.ale.org/pipermail/ale/attachments/20120707/ae341264/attachment.html 