Home page logo
/

nanog logo nanog mailing list archives

Re: Inaccessible network from Verizon, accessible elsewhere.
From: Jonathan Lassoff <jof () thejof com>
Date: Sun, 11 Dec 2011 12:56:52 -0800

On Sat, Dec 10, 2011 at 11:49 AM, NetSecGuy <netsecguy () gmail com> wrote:

I have a Linode VPS in Japan that I can't access from Verizon FIOS,
but can access from other locations.  I'm not sure who to blame.

The host, 106.187.34.33, is behind the gateway 106.187.34.1:

From FIOS to 106.187.34.1  (this works).

traceroute to 106.187.34.1 (106.187.34.1), 64 hops max, 52 byte packets

 4  so-6-1-0-0.phil-bb-rtr2.verizon-gni.net (130.81.199.4)  9.960 ms
9.957 ms  6.666 ms
 5  so-8-0-0-0.lcc1-res-bb-rtr1-re1.verizon-gni.net (130.81.17.3)
12.298 ms  13.463 ms  13.706 ms
 6  0.ae2.br1.iad8.alter.net (152.63.32.158)  14.571 ms  14.372 ms
 14.003 ms
 7  204.255.169.218 (204.255.169.218)  14.692 ms  14.759 ms  13.670 ms
 8  sl-crs1-dc-0-1-0-0.sprintlink.net (144.232.19.229)  13.077 ms
12.577 ms  14.954 ms
 9  sl-crs1-nsh-0-5-5-0.sprintlink.net (144.232.18.200)  31.443 ms
   sl-crs1-dc-0-5-3-0.sprintlink.net (144.232.24.37)  33.005 ms
   sl-crs1-nsh-0-5-5-0.sprintlink.net (144.232.18.200)  31.507 ms
10  sl-crs1-kc-0-0-0-2.sprintlink.net (144.232.18.112)  57.610 ms
58.322 ms  59.098 ms
11  otejbb204.kddnet.ad.jp (203.181.100.45)  196.063 ms
   otejbb203.kddnet.ad.jp (203.181.100.13)  188.846 ms
   otejbb204.kddnet.ad.jp (203.181.100.21)  195.277 ms
12  cm-fcu203.kddnet.ad.jp (124.215.194.180)  214.760 ms
   cm-fcu203.kddnet.ad.jp (124.215.194.164)  198.925 ms
   cm-fcu203.kddnet.ad.jp (124.215.194.180)  200.583 ms
13  124.215.199.122 (124.215.199.122)  193.086 ms *  194.967 ms

This does not work from FIOS:

traceroute to 106.187.34.33 (106.187.34.33), 64 hops max, 52 byte packets

 4  so-6-1-0-0.phil-bb-rtr2.verizon-gni.net (130.81.199.4)  34.229 ms
8.743 ms  8.878 ms
 5  so-8-0-0-0.lcc1-res-bb-rtr1-re1.verizon-gni.net (130.81.17.3)
15.402 ms  13.008 ms  14.932 ms
 6  0.ae2.br1.iad8.alter.net (152.63.32.158)  13.325 ms  13.245 ms
 13.802 ms
 7  204.255.169.218 (204.255.169.218)  14.820 ms  14.232 ms  13.491 ms
 8  lap-brdr-03.inet.qwest.net (67.14.22.78)  90.170 ms  92.273 ms
 145.887 ms
 9  63.146.26.70 (63.146.26.70)  92.482 ms  92.287 ms  94.000 ms
10  sl-crs1-kc-0-0-0-2.sprintlink.net (144.232.18.112)  58.135 ms
58.520 ms  58.055 ms
11  otejbb203.kddnet.ad.jp (203.181.100.17)  205.844 ms
   otejbb204.kddnet.ad.jp (203.181.100.25)  189.929 ms
   otejbb203.kddnet.ad.jp (203.181.100.17)  204.846 ms
12  sl-crs1-oro-0-1-5-0.sprintlink.net (144.232.25.77)  87.229 ms
   sl-crs1-oro-0-3-3-0.sprintlink.net (144.232.25.207)  88.796 ms  88.717
ms
13  124.215.199.122 (124.215.199.122)  193.584 ms  202.208 ms  192.989 ms
14  * * *

Same IP from different network:

traceroute to 106.187.34.33 (106.187.34.33), 30 hops max, 60 byte packets

 6  ae-8-8.ebr2.Washington1.Level3.net (4.69.134.105)  2.230 ms  1.847
ms  1.938 ms
 7  ae-92-92.csw4.Washington1.Level3.net (4.69.134.158)  2.010 ms
1.985 ms ae-62-62.csw1.Washington1.Level3.net (4.69.134.146)  1.942 ms
 8  ae-94-94.ebr4.Washington1.Level3.net (4.69.134.189)  12.515 ms
ae-74-74.ebr4.Washington1.Level3.net (4.69.134.181)  12.519 ms  12.507
ms
 9  ae-4-4.ebr3.LosAngeles1.Level3.net (4.69.132.81)  65.957 ms
65.958 ms  66.056 ms
10  ae-83-83.csw3.LosAngeles1.Level3.net (4.69.137.42)  66.063 ms
ae-93-93.csw4.LosAngeles1.Level3.net (4.69.137.46)  65.985 ms
ae-63-63.csw1.LosAngeles1.Level3.net (4.69.137.34)  66.026 ms
11  ae-3-80.edge2.LosAngeles9.Level3.net (4.69.144.143)  66.162 ms
66.160 ms  66.238 ms
12  KDDI-AMERIC.edge2.LosAngeles9.Level3.net (4.53.228.14)  193.317 ms
 193.447 ms  193.305 ms
13  lajbb001.kddnet.ad.jp (59.128.2.101)  101.544 ms  101.543 ms
lajbb002.kddnet.ad.jp (59.128.2.185)  66.563 ms
14  otejbb203.kddnet.ad.jp (203.181.100.13)  164.217 ms  164.221 ms
 164.330 ms
15  cm-fcu203.kddnet.ad.jp (124.215.194.164)  180.350 ms
cm-fcu203.kddnet.ad.jp (124.215.194.180)  172.779 ms
cm-fcu203.kddnet.ad.jp (124.215.194.164)  185.824 ms
16  124.215.199.122 (124.215.199.122)  175.703 ms  175.700 ms  168.268 ms
17  li377-33.members.linode.com (106.187.34.33)  174.381 ms  174.383
ms  174.368 ms


In doing a little probing right now, from various source addresses, I'm
unable to reproduce the problem.

I've seen failures similar to this one (where the source address matters;
some work, some don't) when multi-port LAGs or ECMP paths have a single
link in them fail, but are still detected and forwarded over as if it was
up. This can happen, for example, if you run a LAG with no channeling
protocol (like LACP or PAGP), that hashes source and destination IPs to
pick a link (to ensure consistent paths per-path, and with ports,
per-flow). If one of those links fails in the underlying media or physical
path, but the link is still detected as up, packets to some IPs (but not
others) will just drop on the flor.

Now, in this particular case, it doesn't seem like the path to both
destinations seem like they even take the same path (so that previous
hypothesis is pure conjecture). Perhaps routes were actively shifting
between KDDI and Sprint (which would explain weird AS paths like
VZB->Qwest->Sprint->KDDI->Sprint)? The IP space belongs to KDDI and is
originated by them, and they're just allocating it for Linode's use.



The last hop is KDDI, but things work from via Level3 and not sprint.
Linode blames Verizon, but I'm not seeing how it's them.


Honestly, I'd see if Linode can work with KDDI to make sure they're
announcing (and others are receiving and routing) to that IP space as
intended.

There's not enough info here to point fingers, but I would think that they
would be the organization most empowered to do anything about this.

Cheers,
jof


  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]