Home page logo

nanog logo nanog mailing list archives

Re: IPv4 Address Exhaustion Effects on the Earth
From: Jim Gettys <jg () freedesktop org>
Date: Wed, 06 Apr 2011 18:43:30 -0400

On 04/05/2011 06:17 PM, Jared Mauch wrote:
On Apr 5, 2011, at 6:07 PM, Jim Gettys wrote:

On 04/05/2011 05:59 PM, Michael Proto wrote:
On Tue, Apr 5, 2011 at 5:38 PM, Jared Mauch<jared () puck nether net>   wrote:
On Apr 4, 2011, at 4:30 PM, Jim Gettys wrote:

Note that the paper "Characterizing Residential Broadband Networks" by Dischinger, et. al. indicates that a large 
fraction (in their 2 year old sample, 30% or so) of broadband head ends are running without RED, and should be doing so if at all 
possible; alternatives are years out by the time they are tested and deployed, and operators running without it in congested 
systems are inflicting pain on their customers.
Something I've observed is if you are sending data 'upstream' on the cable modem setup i have (16 down/ 2 up) and you saturate the 
upstream, the buffering destroys any downstream capability at the same time.  I'm not even sure where to start diagnosing to explaining this to the 
carrier involved, as this isn't the desired behavior of a "business class" service.

- Jared

Isn't this just a case or prioritizing outbound ACKs?


Nope.  Your acks get delayed to what you are sending upstream, behind the downstream traffic.

Bufferbloat hurts both directions, once saturation occurs and your latencies start to go up.

Note that on many of these links, the RTT becomes (literally) as though you are half way (or further than) the moon.

I sent a private reply, but I guess i'll post some of it here:

1) there are no ways to identify the devices doing the buffering and/or drop counts

This isn't really true: you basically do "ping" combined with "traceroute" while saturating a path. The hop where there is unexpected additional latency not present when the you don't saturate the path is the culprit.

You can't easily figure out where inside a bottleneck the buffering is, unless you have some way to monitor or control the buffering inside it (e.g. twist the transmit queue or transmit rings in Linux, as an example).

2) I can obviously feed the cable modem much faster on the lan vs what it can send upstream

Doing things like rate-limiting/QoS are merely just papering over the problem.

Papering over the problem isn't all bad, if it allows you to hide egregious size buffers in a bottleneck link over which you'd otherwise have no control.

It allows me to take my latency/jitter on my home cable service from about 400ms down to 10ms (at some cost in bandwidth). This means I actually can have voip or other latency sensitive applications work (so long as I ensure that the broadband link is the bottleneck; if you home wireless network's effective bandwidth drops below that of your broadband, then the bottleneck becomes the 802.11 hop, and you'll see queues in your host and home router, rather than in the broadband hop. Notice that this means with symmetric 25/25 FIOS service, you get bufferbloat in your host and wireless router, as I did (actual data transfer rates of 802.11g is only about 20Mbps, not the 54 signalling rate marketed).

I would take a T1 and rate-limit it to 1.2Mb/s for TCP to allow VoIP to work.  Junipers can buffer up to 1 second on 
these low-speed interfaces, which obviously creates the problems you describe.

Bufferbloat is (nearly) everywhere.

You'd better see if you can run RED or some AQM. The issues with RED revolve around the fact that it is a flawed algorithm that requires proper tuning to be effective, and it can't handle situations such as wireless or broadband with time variable behaviour such as Comcast's Powerboost.

It is exactly the fact that RED requires tuning that means that it is often not enabled when it should be: but it is often the only tool we've got right now.

There are a lot more problems with the gateway devices, such as the forcible dns proxy that exists.

Right now, the home router market is a cesspool of scum and villainy. Our best hope is the rebel alliance; I think we'll get OpenWRT straightened out long before the commercial vendors do.
                - Jim

  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]