Re: D/DoS mitigation hardware/software needed.

[Joe Greco <jgreco () ns sol net>]

> > Then you need to get rid of that '90's antique web server and get
> > something modern.  When you say "interrupt-bound hardware," all you
> > are doing is showing that you're not familiar with modern servers
> > and quality operating systems that are designed to mitigate things
> > like DDoS attacks.
>
> "Modern" servers?   IP is processed in the kernel on web servers,
> regardless of OS.  Have you configured a kernel lately?

Yes, pretty much every time I install a server.

> Noticed there
> are ~3,000 lines in the Linux config file alone?  

Well, that explains a lot.

% wc -l /sys/i386/conf/WEBX4
     324 /sys/i386/conf/WEBX4

I probably haven't noticed that there are ~3,000 lines in the Linux
config file alone because I use a different OS; ~3,000 lines of config
would just be another example of why I generally consider Linux to be
a little broken.  I can see why admins would be hesitant to challenge
such a thing.

> _Lots_ of device
> drivers in there, which are interrupt driven and have to be timeshared.
> No servers I know do realtime processing (RT kernels don't) or process IP
> in ASICs.

Roger, meet FreeBSD.  FreeBSD, meet Roger.  FreeBSD, would you please show
Roger how IP is handled without excessive interrupts?

% systat -vm (snipped from larger display)
Interrupts
2208 total
     stray irq7
     mux irq9
     em5 irq5
  85 ata0 irq14
     mux irq11
     fdc0 irq6
     atkbd0 irq
     sio0 irq4
1995 clk irq0
 128 rtc irq8

% netstat 1
            input        (Total)           output
   packets  errs      bytes    packets  errs      bytes colls
     58991     0   54547321      58975     0   54523849     0
     59492     0   58297208      59475     0   58388027     0
     65828     0   62105928      65856     0   62081922     0
     60257     0   56781863      60219     0   56809674     0
     62547     0   61254034      62583     0   61231514     0
     58188     9   55536734      58103     0   55560822     0
     73870     0   70245952      73959     0   70223249     0
     61436     0   58766122      61429     0   58786292     0
     61390     0   59050710      61336     0   59029298     0
     61447     0   58701312      61502     0   58725356     0
     63934     0   60801413      63932     0   60777621     0
     60187     0   56724030      60189     0   56751946     0
     60247     0   55544082      60036     0   55522162     0
     66472     0   63061572      66635     0   63033232     0
     66415     0   62876955      66438     0   62854488     0
     66612     0   63270235      66355     0   63335538     0
     66020     0   60478426      66293     0   60454874     0
     67696     0   63512069      67692     0   63534500     0
     66342     0   60462142      66353     0   60439239     0

That's 60Kpps being handled with 2K interrupts per second.  It'll be
2K interrupts per second at 0pps or 200Kpps or whatever.

% ipfw l | wc -l
     620

It's doing nontrivial amounts of firewalling while doing this.

% top
last pid: 83148;  load averages:  0.31,  0.28,  0.23   up 459+08:00:24 12:00:33
51 processes:  3 running, 42 sleeping, 6 stopped
CPU states: 14.8% user,  0.0% nice, 19.1% system, 13.3% interrupt, 52.7% idle

% cat /var/run/dmesg.boot
[...]
CPU: Intel(R) Pentium(R) 4 CPU 3.00GHz (2994.90-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0xf41  Stepping = 1
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,C
MOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
[...]

Ewww, but it *is* a 2004-vintage Pentium Prescott CPU on a legacy PCI mobo, 
so it is actually a little disadvantaged compared to modern hardware.


> What configurations of Linux / BSD / Solaris / etc does web / email / ntp
> / sip / iptables / ipfw / ... and doesn't have issues with kernel
> locking?

That's like saying "what cars cannot be crashed into a wall."  A much
better question is "what combination of driver and vehicle can I get
that significantly reduces the chances of my being involved in a crash."
Driver is important because even the best vehicle can be driven into a
wall; vehicle is important because even the best driver is severely
limited by a decrepit old car.  It's when you get a great driver in a
great vehicle that you get the good results.

> Test it on your own servers by mounting a damaged DVD on the
> root directory, and dd'ing it to /dev/null.  Notice how the ATA/SATA/SCSI
> driver impacts the latency of everything on the system. 

As soon as a remote attacker is able to insert a damaged DVD into one
of my servers (maybe via specially crafted IP options in a TCP packet?),
you will witness my posterior emit a large number of blocks of ceramic 
material (used in masonry construction).  Until then, I am unfazed by
this because it isn't particularly relevant to the discussion.  I can
cause excessive latency simply by switching off gear too.

I *strongly* suggest you go and look over

http://info.iet.unipi.it/~luigi/polling/

/and note its date/ before you compose any reply; device polling has been
around for a *long* time and its usefulness as a DDoS mitigator in the
server arena is hard to refute.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.