[NANOG] Verizon (Fios) Packet Loss | Philadelphia Metro Area | Possible Routing Issue?

["alexander.lembesis--- via NANOG" <nanog () lists nanog org>]

Hello, fellow NANOG members,

Is there anyone from Verizon on the list that could take a look at a possible routing issue on the Fios network that 
could be causing loss?  I have two locations, each with 1G service.
Location 1 is in Quakertown, PA & has a dynamic IP on the 96.227.54.0/24 block
Location 2 is in Chalfont, PA & has a dynamic IP on the 108.16.195.0/24 block
They are ~11miles apart as the crow flies.  I have some EdgeConnect SDWAN appliances connected at these locations and 
they're measuring a variable amount of loss between them (more during peak hours).  I have checked everything I can on 
my end and posting here as a last resort.

Traceroute from Quakertown to Chalfont seems to hop through two LCR nodes in Washington, DC.                            
           
 1. dog-coresw01.xxxxxxxxx.net             
 2. ixl0-dog-opnsense01.xxxxxxxxx.NET       
 3. lo0-100.PHLAPA-VFTTP-346.verizon-gni.net
 4. B3346.PHLAPA-LCR-21.verizon-gni.net     
    B3346.PHLAPA-LCR-22.verizon-gni.net
 5. G1-7-2-4.WASHDC-LCR-21.verizon-gni.net  
    G1-7-2-3.WASHDC-LCR-21.verizon-gni.net
 6. Chalfont, PA 108.16.195.x

Unless the PTR records on hop #5 are incorrect, it seems odd that traffic from Quakertown, PA has to travel all the way 
down to Washington DC, then come back up to Pennsylvania.

Traceroute from the opposite direction (Chalfont to Quakertown) does not show the WASHDC LCR nodes.
1. 192.168.138.1                                   
2. ixl0-wpv-opnsense.xxxxxxxxx.NET                 
3. lo0-100.PHLAPA-VFTTP-354.verizon-gni.net        
4. (waiting for reply)
5. Bundle-Ether1000.PHLAPA-LCR-21.VERIZON-GNI.NET  
   Bundle-Ether1000.PHLAPA-LCR-22.VERIZON-GNI.NET
   Bundle-Ether1200.PHLAPA-LCR-22.VERIZON-GNI.NET
   HundredGigE2-0-0-3.PHLAPA-LCR-21.VERIZON-GNI.NET
   HundredGigE2-0-0-0.PHLAPA-LCR-21.VERIZON-GNI.NET
   Bundle-Ether800.PHLAPA-LCR-22.VERIZON-GNI.NET
   HundredGigE2-0-0-3.PHLAPA-LCR-22.VERIZON-GNI.NET
6. ae204-0.PHLAPA-VFTTP-346.verizon-gni.net        
   ae203-0.PHLAPA-VFTTP-346.verizon-gni.net
7. Quakertown, PA 96.227.54.x

I'm out of ideas on what could be causing the packet loss.  If someone from Verizon could take a look, that would be 
GREATLY appreciated.

Thank you!!!
_______________________________________________
NANOG mailing list 
https://lists.nanog.org/archives/list/nanog () lists nanog org/message/2DD4GF7HBJN3CJ5WYHGMOMILI6ZHEGKR/