Re: how about a well-known DOWNGRADE BGP Community? (Was: RTBH Support Across the Industry)

[Lukas Tribus via NANOG <nanog () lists nanog org>]

Hello,

I would be very happy to see this feature implemented at transit
providers, because it takes the guesswork out of RTBH and allows
visibility into the attack patterns without sacrificing not attacked
destinations.


But ... TL;DR: know the routing platform your are implementing this on
inside and out.

Take an ASR9000 for example. A basic implementation that sets a
ingress qos-group/cos/dscp/exp bit for the attacked /32 and has an
egress queue configurations which deprioritizes scavenger traffic can
do nothing in a DoS situation.

Let's say the core is Nx100G, the customer has a single 10G (on a
different line card), and a 40G attack goes through from the core
towards a customer IP, customer enables scavenger marking for the
attacked /32, and then nothing happens.

The customers continues to see heavy packet loss and huge latency even
on IP addresses that are not under attack and egress QoS doesn't seem
to do that much considering the attack size.


Why is this happening?

HoLB avoidance with Virtual Output Queues and back pressure [1].

The ingress line card (with Nx100G core links) on the 9k has a virtual
output queue for each egress 10G entity.
The ingress line card does NOT apply egress QoS port settings of
course, that's the job of the egress line card.

All traffic destined for the 10G customer port including attack
traffic is treated equally by the ingress line card, meaning it is
queued at 10G without consideration for egress port QoS settings
before entering the fabric. That is why the egress line cards receives
a 10G limited stream of traffic (per ingress LC), which the DoS
obviously dominates and the actual configured egress port QoS setting
has nothing left to do.

VoQ QoS is extremely limited (4 queues: default, multicast, priority 1
and priority 2). IIRC priority queues are strict (policer) and cause
back pressure to fabric interface ASICs when congested (so inverting
the logic and putting normal traffic into a priority queue would never
work, not that anybody would do something like this).

I really don't know how you'd implement this on a 9k, unless a fifth
scavenger/below normal queue for VoQ would be implemented on the
ingress LC (and this would be a per LC HW dependant feature).

When you try to protect your customer from a DoS with an egress port
ACL (like drop all IP frag + UDP traffic to this /32), a similar thing
happens: TX traffic dips below normal on the customer port. Again
because *all* traffic is queued unconditionally before the ACL on the
egress line card can make a difference and then, the ACL actually
drops (and therefore hides) the attack traffic, leaving less than
normal but still heavily congested legitimate traffic left for TX.

Of course if the traffic never goes to the fabric (ingress and egress
port on the same LC), this is not an issue or when you have 8
linecards and each linecards has 1 core link, all perfectly load
balanced, than you'd probably still get a huge benefit from this.

However this is a lot more complex than just a adding another logic to
the routing policy.


Kind regards,
Lukas

[1] 
https://community.cisco.com/t5/service-providers-knowledge-base/asr9000-xr-understanding-qos-default-marking-behavior-and/ta-p/3128709
_______________________________________________
NANOG mailing list 
https://lists.nanog.org/archives/list/nanog () lists nanog org/message/UJGPNRZYXD6BMHP4AZY3EFMWKBYDMD6S/