mailing list archives
Re: IPv4 Address Exhaustion Effects on the Earth
From: Jim Gettys <jg () freedesktop org>
Date: Wed, 06 Apr 2011 18:43:30 -0400
On 04/05/2011 06:17 PM, Jared Mauch wrote:
On Apr 5, 2011, at 6:07 PM, Jim Gettys wrote:
On 04/05/2011 05:59 PM, Michael Proto wrote:
On Tue, Apr 5, 2011 at 5:38 PM, Jared Mauch<jared () puck nether net> wrote:
On Apr 4, 2011, at 4:30 PM, Jim Gettys wrote:
Note that the paper "Characterizing Residential Broadband Networks" by Dischinger, et. al. indicates that a large
fraction (in their 2 year old sample, 30% or so) of broadband head ends are running without RED, and should be doing so if at all
possible; alternatives are years out by the time they are tested and deployed, and operators running without it in congested
systems are inflicting pain on their customers.
Something I've observed is if you are sending data 'upstream' on the cable modem setup i have (16 down/ 2 up) and you saturate the
upstream, the buffering destroys any downstream capability at the same time. I'm not even sure where to start diagnosing to explaining this to the
carrier involved, as this isn't the desired behavior of a "business class" service.
Isn't this just a case or prioritizing outbound ACKs?
Nope. Your acks get delayed to what you are sending upstream, behind the downstream traffic.
Bufferbloat hurts both directions, once saturation occurs and your latencies start to go up.
Note that on many of these links, the RTT becomes (literally) as though you are half way (or further than) the moon.
I sent a private reply, but I guess i'll post some of it here:
1) there are no ways to identify the devices doing the buffering and/or drop counts
This isn't really true: you basically do "ping" combined with
"traceroute" while saturating a path. The hop where there is unexpected
additional latency not present when the you don't saturate the path is
You can't easily figure out where inside a bottleneck the buffering is,
unless you have some way to monitor or control the buffering inside it
(e.g. twist the transmit queue or transmit rings in Linux, as an example).
2) I can obviously feed the cable modem much faster on the lan vs what it can send upstream
Doing things like rate-limiting/QoS are merely just papering over the problem.
Papering over the problem isn't all bad, if it allows you to hide
egregious size buffers in a bottleneck link over which you'd otherwise
have no control.
It allows me to take my latency/jitter on my home cable service from
about 400ms down to 10ms (at some cost in bandwidth). This means I
actually can have voip or other latency sensitive applications work (so
long as I ensure that the broadband link is the bottleneck; if you home
wireless network's effective bandwidth drops below that of your
broadband, then the bottleneck becomes the 802.11 hop, and you'll see
queues in your host and home router, rather than in the broadband hop.
Notice that this means with symmetric 25/25 FIOS service, you get
bufferbloat in your host and wireless router, as I did (actual data
transfer rates of 802.11g is only about 20Mbps, not the 54 signalling
I would take a T1 and rate-limit it to 1.2Mb/s for TCP to allow VoIP to work. Junipers can buffer up to 1 second on
these low-speed interfaces, which obviously creates the problems you describe.
Bufferbloat is (nearly) everywhere.
You'd better see if you can run RED or some AQM. The issues with RED
revolve around the fact that it is a flawed algorithm that requires
proper tuning to be effective, and it can't handle situations such as
wireless or broadband with time variable behaviour such as Comcast's
It is exactly the fact that RED requires tuning that means that it is
often not enabled when it should be: but it is often the only tool we've
got right now.
Right now, the home router market is a cesspool of scum and villainy.
Our best hope is the rebel alliance; I think we'll get OpenWRT
straightened out long before the commercial vendors do.
There are a lot more problems with the gateway devices, such as the forcible dns proxy that exists.
Re: IPv4 Address Exhaustion Effects on the Earth Alexander Maassen (Apr 02)
IPv4 Address Exhaustion Effects on the Earth Michael Ruiz (Apr 04)
- Re: IPv4 Address Exhaustion Effects on the Earth, (continued)