nanog mailing list archives

Re: Level3 4.2.2.1 and 4.2.2.2 not responding to pings, DNS queries work


From: Warren Kumari <warren () kumari net>
Date: Mon, 13 Jan 2025 08:12:00 -0800

On Mon, Jan 13, 2025 at 10:48 AM, Mel Beckman <mel () beckman org> wrote:


Dan,



That dig tip for identifying the NS phy loc is very nice! That's something
I can put in our support procedures for DNS troubleshooting.



Yup. Note that many authoritative DNS servers also support RFC5001 - "DNS
Name Server Identifier (NSID) Option"
<https://datatracker.ietf.org/doc/rfc5001/> (NSID). This can be really
valuable for figuring out which instance of an Anycast server you are
hitting.

E.g:
$ dig +nsid +edns NS . @b.root-servers.net
[[SNIP]]
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; NSID: 62 31 2d 69 61 64 ("b1-iad")
;; QUESTION SECTION:
;. IN NS

;; ANSWER SECTION:
. 518400 IN NS a.root-servers.net.
. 518400 IN NS b.root-servers.net.
. 518400 IN NS c.root-servers.net.
[[SNIP]]

W




   -mel

------------------------------

*From:* nanog () fleish org <nanog () fleish org>
*Sent:* Monday, January 13, 2025 7:19 AM
*To:* Mel Beckman <mel () beckman org>; sterling.daniel () gmail com <sterling.
daniel () gmail com>
*Cc:* North American Network Operators' Group <nanog () nanog org>
*Subject:* Re: Level3 4.2.2.1 and 4.2.2.2 not responding to pings, DNS
queries work



I’m seeing some of the resolver IPs being filtering from various locations
while responding from others, no doubt due to their use of anycast. I
rarely ping test 4.2.2.2 anymore, having switched to 8.8.8.8 some time ago
which I have a window up running 24x7 as a quick way to detect if my laptop
is having connectivity issues from wherever it currently resides. It’s
usually reliable, but I understand such traffic is the first to get
filtered/dropped when needed so it’s just an initial indicator for me from
which further testing can be performed when needed.


I will also mention it’s possible to detect which NS server/pool/location
you are reaching on Level3’s network via the below dig query. This has been
an invaluable tool over the years as IME they tend to break DNS
sub-delegation at least once or twice a year and the more data you can
provide to them about where the breakage is the faster you can get them to
engage the DNS team to actually fix it vs. arguing with you that it’s not
broken because it "works for them” (facepalm). And at least once I found
they broke resolution on their authoritative name servers (ns1.l3.net or
ns2.l3.net) and again they first told me it wasn’t a problem because DNS
again the broken one(s) would timeout and then get the answer from another,
working server (double-facepalm).




dig +short @4.2.2.2 hostname.bind CH TXT




I’ve also been provided this query to derive the same for Cloudflare’s NS
servers:




dig CHAOS TXT id.server @1.1.1.1 +nsid



And the following for Cleanbrowsing NS servers (these power content
filtering on Unifi networks):



nslookup -type=txt iptest.whois.dnscontest.cleanbrowsing.org






Below are the results I got for Level3 from various locations (pardon the
wall of text). It looks like they’re making some changes to serve DNS
queries off their NTP servers



From Level3 WDC1/McLean:





PING 4.2.2.1 (4.2.2.1) 56(84) bytes of data.



--- 4.2.2.1 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 0.987/0.987/0.987/0.000 ms

"pubntp1.wdc12"

PING 4.2.2.2 (4.2.2.2) 56(84) bytes of data.



--- 4.2.2.2 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 0.926/0.926/0.926/0.000 ms

"cns4.sjo1"

PING 4.2.2.3 (4.2.2.3) 56(84) bytes of data.



--- 4.2.2.3 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 0.799/0.799/0.799/0.000 ms

"pubntp1.wdc12"

PING 4.2.2.4 (4.2.2.4) 56(84) bytes of data.



--- 4.2.2.4 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 0.855/0.855/0.855/0.000 ms

"pubntp2.wdc12"

PING 4.2.2.5 (4.2.2.5) 56(84) bytes of data.



--- 4.2.2.5 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 0.897/0.897/0.897/0.000 ms

"cns4.sjo1"

PING 4.2.2.6 (4.2.2.6) 56(84) bytes of data.



--- 4.2.2.6 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 0.973/0.973/0.973/0.000 ms

"pubntp1.wdc12”




From Level3 SFO1/Sunnyvale:




PING 4.2.2.1 (4.2.2.1) 56(84) bytes of data.



--- 4.2.2.1 ping statistics ---

2 packets transmitted, 0 received, 100% packet loss, time 999ms



"cns3.sjo1"

PING 4.2.2.2 (4.2.2.2) 56(84) bytes of data.



--- 4.2.2.2 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms



"cns2.sjo1"

PING 4.2.2.3 (4.2.2.3) 56(84) bytes of data.



--- 4.2.2.3 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms



"cns1.sjo1"

PING 4.2.2.4 (4.2.2.4) 56(84) bytes of data.



--- 4.2.2.4 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms



"cns2.sjo1"

PING 4.2.2.5 (4.2.2.5) 56(84) bytes of data.



--- 4.2.2.5 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 0.275/0.275/0.275/0.000 ms

"cns3.sjo1"

PING 4.2.2.6 (4.2.2.6) 56(84) bytes of data.



--- 4.2.2.6 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms



"cns3.sjo1”




From Comcast Atlanta:




PING 4.2.2.1 (4.2.2.1) 56(84) bytes of data.



--- 4.2.2.1 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms



"cns2.atl2"

PING 4.2.2.2 (4.2.2.2) 56(84) bytes of data.



--- 4.2.2.2 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 13.442/13.442/13.442/0.000 ms

"cns4.atl2"

PING 4.2.2.3 (4.2.2.3) 56(84) bytes of data.



--- 4.2.2.3 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 13.431/13.431/13.431/0.000 ms

"cns2.atl2"

PING 4.2.2.4 (4.2.2.4) 56(84) bytes of data.



--- 4.2.2.4 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 13.632/13.632/13.632/0.000 ms

"cns2.atl2"

PING 4.2.2.5 (4.2.2.5) 56(84) bytes of data.



--- 4.2.2.5 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 12.656/12.656/12.656/0.000 ms

"cns3.atl2"

PING 4.2.2.6 (4.2.2.6) 56(84) bytes of data.



--- 4.2.2.6 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 12.041/12.041/12.041/0.000 ms

"cns4.atl2”




From AT&T Atlanta:




PING 4.2.2.1 (4.2.2.1) 56(84) bytes of data.



--- 4.2.2.1 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms



"cns2.atl2"

PING 4.2.2.2 (4.2.2.2) 56(84) bytes of data.



--- 4.2.2.2 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms



"pubntp1.atl2"

PING 4.2.2.3 (4.2.2.3) 56(84) bytes of data.



--- 4.2.2.3 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms



"cns3.atl2"

PING 4.2.2.4 (4.2.2.4) 56(84) bytes of data.



--- 4.2.2.4 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 15.430/15.430/15.430/0.000 ms

"pubntp1.atl2"

PING 4.2.2.5 (4.2.2.5) 56(84) bytes of data.



--- 4.2.2.5 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms



"cns3.atl2"

PING 4.2.2.6 (4.2.2.6) 56(84) bytes of data.



--- 4.2.2.6 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 16.103/16.103/16.103/0.000 ms

"cns4.atl2”




From AT&T SF Bay Area Peninsula:




PING 4.2.2.1 (4.2.2.1) 56(84) bytes of data.



--- 4.2.2.1 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 7.880/7.880/7.880/0.000 ms

"cns3.sjo1"

PING 4.2.2.2 (4.2.2.2) 56(84) bytes of data.



--- 4.2.2.2 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms



"cns2.sjo1"

PING 4.2.2.3 (4.2.2.3) 56(84) bytes of data.



--- 4.2.2.3 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 6.888/6.888/6.888/0.000 ms

"cns3.sjo1"

PING 4.2.2.4 (4.2.2.4) 56(84) bytes of data.



--- 4.2.2.4 ping statistics ---

2 packets transmitted, 0 received, 100% packet loss, time 1000ms



"cns4.sjo1"

PING 4.2.2.5 (4.2.2.5) 56(84) bytes of data.



--- 4.2.2.5 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 7.173/7.173/7.173/0.000 ms

"cns1.sjo1"

PING 4.2.2.6 (4.2.2.6) 56(84) bytes of data.



--- 4.2.2.6 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms



"cns2.sjo1”




From DRFortress Honolulu:




PING 4.2.2.1 (4.2.2.1) 56(84) bytes of data.



--- 4.2.2.1 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms



"cns3.sjo1"

PING 4.2.2.2 (4.2.2.2) 56(84) bytes of data.



--- 4.2.2.2 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms



"cns3.sjo1"

PING 4.2.2.3 (4.2.2.3) 56(84) bytes of data.



--- 4.2.2.3 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 54.233/54.233/54.233/0.000 ms

"cns3.sjo1"

PING 4.2.2.4 (4.2.2.4) 56(84) bytes of data.



--- 4.2.2.4 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms



"cns3.sjo1"

PING 4.2.2.5 (4.2.2.5) 56(84) bytes of data.



--- 4.2.2.5 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 54.195/54.195/54.195/0.000 ms

"cns3.sjo1"

PING 4.2.2.6 (4.2.2.6) 56(84) bytes of data.



--- 4.2.2.6 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms



"cns4.sjo1”




From Hawaiian Telecom Honolulu (oddly from here I get no response to the
hostname.bind dig queries):




PING 4.2.2.1 (4.2.2.1) 56(84) bytes of data.



--- 4.2.2.1 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 54.727/54.727/54.727/0.000 ms

PING 4.2.2.2 (4.2.2.2) 56(84) bytes of data.



--- 4.2.2.2 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms



PING 4.2.2.3 (4.2.2.3) 56(84) bytes of data.



--- 4.2.2.3 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms



PING 4.2.2.4 (4.2.2.4) 56(84) bytes of data.



--- 4.2.2.4 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms



PING 4.2.2.5 (4.2.2.5) 56(84) bytes of data.



--- 4.2.2.5 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 56.765/56.765/56.765/0.000 ms

PING 4.2.2.6 (4.2.2.6) 56(84) bytes of data.



--- 4.2.2.6 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms




From Unitas Seacacus:




PING 4.2.2.1 (4.2.2.1) 56(84) bytes of data.



--- 4.2.2.1 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 2.062/2.062/2.062/0.000 ms

"cns2.nyc6"

PING 4.2.2.2 (4.2.2.2) 56(84) bytes of data.



--- 4.2.2.2 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 2.012/2.012/2.012/0.000 ms

"cns2.nyc6"

PING 4.2.2.3 (4.2.2.3) 56(84) bytes of data.



--- 4.2.2.3 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 1.651/1.651/1.651/0.000 ms

"cns3.nyc6"

PING 4.2.2.4 (4.2.2.4) 56(84) bytes of data.



--- 4.2.2.4 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 2.055/2.055/2.055/0.000 ms

"cns2.nyc6"

PING 4.2.2.5 (4.2.2.5) 56(84) bytes of data.



--- 4.2.2.5 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 2.102/2.102/2.102/0.000 ms

"cns3.nyc6"

PING 4.2.2.6 (4.2.2.6) 56(84) bytes of data.



--- 4.2.2.6 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 1.626/1.626/1.626/0.000 ms

"cns3.nyc6”




From Verizon FIOS New York (oddly from here I get no response to the
hostname.bind dig queries):






PING 4.2.2.1 (4.2.2.1) 56(84) bytes of data.



--- 4.2.2.1 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 4.355/4.355/4.355/0.000 ms

PING 4.2.2.2 (4.2.2.2) 56(84) bytes of data.



--- 4.2.2.2 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 3.764/3.764/3.764/0.000 ms

PING 4.2.2.3 (4.2.2.3) 56(84) bytes of data.



--- 4.2.2.3 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 4.053/4.053/4.053/0.000 ms

PING 4.2.2.4 (4.2.2.4) 56(84) bytes of data.



--- 4.2.2.4 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 3.394/3.394/3.394/0.000 ms

PING 4.2.2.5 (4.2.2.5) 56(84) bytes of data.



--- 4.2.2.5 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 3.442/3.442/3.442/0.000 ms

PING 4.2.2.6 (4.2.2.6) 56(84) bytes of data.



--- 4.2.2.6 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 4.944/4.944/4.944/0.000 ms




From Allied Telecom in Washington DC:





PING 4.2.2.1 (4.2.2.1) 56(84) bytes of data.



--- 4.2.2.1 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 1.683/1.683/1.683/0.000 ms

"pubntp1.wdc12"

PING 4.2.2.2 (4.2.2.2) 56(84) bytes of data.



--- 4.2.2.2 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 1.592/1.592/1.592/0.000 ms

"pubntp2.wdc12"

PING 4.2.2.3 (4.2.2.3) 56(84) bytes of data.



--- 4.2.2.3 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 1.776/1.776/1.776/0.000 ms

"cns1.wdc12"

PING 4.2.2.4 (4.2.2.4) 56(84) bytes of data.



--- 4.2.2.4 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 1.810/1.810/1.810/0.000 ms

"pubntp2.wdc12"

PING 4.2.2.5 (4.2.2.5) 56(84) bytes of data.



--- 4.2.2.5 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 1.729/1.729/1.729/0.000 ms

"cns1.wdc12"

PING 4.2.2.6 (4.2.2.6) 56(84) bytes of data.



--- 4.2.2.6 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 1.432/1.432/1.432/0.000 ms

"cns4.sjo1”




-T

















On Jan 13, 2025, at 07:00, nanog-request () nanog org wrote:


Message: 6
Date: Mon, 13 Jan 2025 04:24:50 +0000
From: Mel Beckman <mel () beckman org>
To: Daniel Sterling <sterling.daniel () gmail com>
Cc: Jerry Cloe <jerry () jtcloe net>, "nanog () nanog org" <nanog () nanog org>
Subject: Re: Level3 4.2.2.1 and 4.2.2.2 not responding to pings, DNS
queries work
Message-ID: <1E72A509-881F-453A-B5D4-7AF8FDB44C4E () beckman org>
Content-Type: text/plain; charset="utf-8"

Dan,

Thanks! I had never read that before. But that makes sense.

-mel

On Jan 12, 2025, at 8:22?PM, Daniel Sterling <sterling.daniel () gmail com>
wrote:

?
Seems like these IPs not responding to ping is not unusual, as per https:/
/www.reddit.com/r/sysadmin/comments/11syv2e/
google_dns_8888_dropping_pings_like_crazy_today/

"Google has stated multiple times before as has Level3/CenturyLink/Lumen
that 4.2.2.1 and 8.8.8.8 should not be used for ping checks and they will
drop packets when under load or if they notice too much activity from a
single IP"

-- Dan

On Sun, Jan 12, 2025 at 11:03?PM Mel Beckman <mel () beckman org<mailto:mel@
beckman.org <mel () beckman org>>> wrote:
Still not pinging from Frontier, Lumen, AT&T, or Verizon networks


-mel

On Jan 12, 2025, at 4:13?PM, Jerry Cloe <jerry () jtcloe net<mailto:jerry@
jtcloe.net <jerry () jtcloe net>>> wrote:

?
O:\>ping 4.2.2.1

Pinging 4.2.2.1 with 32 bytes of data:
Reply from 4.2.2.1<http://4.2.2.1>: bytes=32 time=36ms TTL=56
Reply from 4.2.2.1<http://4.2.2.1>: bytes=32 time=44ms TTL=56
Reply from 4.2.2.1<http://4.2.2.1>: bytes=32 time=36ms TTL=56
Reply from 4.2.2.1<http://4.2.2.1>: bytes=32 time=38ms TTL=56

Ping statistics for 4.2.2.1<http://4.2.2.1>:
   Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
   Minimum = 36ms, Maximum = 44ms, Average = 38ms

Same for 4.2.2.2



-----Original message-----
From: Mel Beckman <mel () beckman org<mailto:mel () beckman org
<mel () beckman org>>>
Sent: Sun 01-12-2025 06:07 pm
Subject: Level3 4.2.2.1 and 4.2.2.2 not responding to pings, DNS queries
work
To: nanog () nanog org<mailto:nanog () nanog org <nanog () nanog org>>;
I noticed that Level3 open DNS 4.2.2.1 and 4.2.2.2 stopped responding to
ping today. They are responding to DNS queries however.

Does anyone know if this filtering is going to be permanent?

-mel beckman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20250113/
eb801d1d/attachment-0001.html>



Current thread: