Re: Distributed Router Fabrics

[joel jaeggli <joelja () bogus com>]

On 12/26/24 14:46, Randy Bush wrote:
> > In a distributed fabric, where is the traditional control plane run?
> > Say I've got 100 BGP sessions of upstream,peer, and downstream across
> > ten routers. Is each pizza box grinding this out on its own, or is the
> > work done on the x86 box mentioned in the larger installations?
> one way to think of it is that each pizza box (customer facing ports)
> recognizes control plane messages (e.g. port 179) and "punts" them to
> the control plane box, aka routing engine.

This is similar to the way I think about it.

In  a router(switch) with a bunch of linecards  (1 or more) there are a 
set of match rules (ACLs if you will) which match traffic bound for the 
control-plane and forward them via a management port to the control 
plane cpu, conveniently this is also where you implement your 
control-plane-protection.  If you substitute  ethernet switch for 
line-card, broadly you are an in a similar place conceptually if the 
main control plane processor is at some remove from the switch that is 
now a line card. at some point you need to encapsulate the control-plane 
messages because the managment next-hop is remote rather than local and 
the neighbor is also a switch.

For me, the realization a decade ago was that enclosing a large number 
of asics in sheet metal with the assiciated midplane and glue logic was 
a higher capital risk and reduced the size of the addressable market vs 
smaller switches with could be assembled piecemeal into a larger switch. 
The white box vendors and the ODMs are loath to build something that has 
limited market addressability so it becomes more attractive over time to 
build large assemblies of box rather than large boxes that can leverage 
the atom of switch that  the 1/2ru pizza box can enclose and just buy 
more of them. That said the maximum radix of a single large switch asic 
right now is 512 * 100G ports.  so you need to be able to build a box 
that can enclose that many ports or the 64 x 800G, that, that maps to 
which is still a very hefty pizza box.


> randy