Very Large IDS implementations (was Re: RE: testing ids)

[Bennett Todd <bet () rahul net>]

2003-03-17T12:26:32 Benjamin Hippler:
> Has anyone experience with very large IDS implementations?

Me! Me! I have! Pick Me! (jumping up and down, waving hand)

It depends on what you mean by "very large IDS". There's one
direction of "large" that can't be tackled without some heavy-duty
engineering and some expensive kit: that's IDS monitoring a
high-bandwidth (>>200Mbps) links. For folks who want to do
everything IDSish with open source snort on commodity hardware, the
only way I know to hit big bandwidth is with a farm of snorters fed
via Toplayer switch. There may be other ways, but that's the one
I've heard about. Without a toplayer, snort can handle c. 50Mbps
untuned, as much as 250-300Mbps tuned on PCI bus, and possibly as
high as 500-600Mbps on PCIx with perfect tuning.

What I have done is a pretty big deployment with lots of servers
(>>50 sensors) world-wide. We planned things so each sensor had
under 50Mbps it was responsible for, and that brought it handily
into the reach of untuned snort on cheap 1U rackmounts, O Joy!.

In this space, the thing to do is pick your platforms and strategies
to match your experience and expertise and existing tooling.

We did snort on Red Hat 7.3, in-house rpmmed. The (separate)
snort-sigs rpm include, open-coded in the spec file, all our tuning,
expressed as a series of edits on various config files, with
dispatching to the appropriate config file by lookup in the init
script, along the lines of

        cfg=/etc/snort/snort.conf
        f=/etc/snort/`hostname`-snort.conf
        test -f $f && cfg=$f

Add some additional tuning for finishing off the other per-device
tuning (bonding appropriate interfaces, setting the mgmt interface
IP addr, netmask, defaultrouter; setting the logserver) and
automatic package update, and you've got an enterprise appliance.

The key here is to pick a basis for building enterprise appliances
for which you have sufficient in-house expertise. If you don't have
it, then you want to buy a commercial solution. SourceFire would be
my first choice in that space.

> Where can I find whitepapers, best practices,.. about this topic?

Beats me, I never looked for any. There are probably some relevent
papers on snort.org, and I wouldn't be surprised if there weren't
also some helpful material at securityfocus.org.

-Bennett

Attachment: _bin
Description: