Re: Sites unreachable while traversing Dallas IXP

[William Herrin via NANOG <nanog () lists nanog org>]

On Sat, Sep 27, 2025 at 7:31 PM Bruce Wainer via NANOG
<nanog () lists nanog org> wrote:
> Excuse my ignorance about this IXP and your equipment, but is
> Micro-BFD (RFC 7130) supported? And if so, is it enabled or can you
> enable it? While configuration wise it will use the single IP
> addresses of the aggregate, separate BFD instances are set up for each
> underlying link and will confirm whether Layer 3 is working on that
> point-to-point connection.

Hi Bruce,

I'm also not familiar with this particular IXP but generally with IXPs
we're not talking about point to point connections. The multiple
participants' routers are part of a shared layer-2 fabric (a switch or
switches) over which they trade layer-3 packets directly with each
other. The route advertisements may transit the route servers but the
routed packets do not.

You can get into some really finicky errors where both participants
successfully talk to the route server and thereby exchange routes, but
for one reason or another can't get packets back and forth to each
other. Bonded circuits (LAGs) add complexity which makes
troubleshooting that much harder.

If it were me, I would have considered building this connection
differently. For speed, I'd have chosen a 100G link instead of two 10G
links. Had my objective been reliability, I'd have built that at layer
3 instead of layer 2 -- two routers each with its own 10G link, and
then done some balancing of the advertised routes. But in all fairness
to Andy, I don't have anywhere near complete information here and the
details matter a lot.

Regards,
Bill Herrin

-- 
William Herrin
bill () herrin us
https://bill.herrin.us/
_______________________________________________
NANOG mailing list 
https://lists.nanog.org/archives/list/nanog () lists nanog org/message/RXKZDTZBXR4VTGE7R6E55FHAC7QEVDHY/