RE: RE: IDS taps in a switched network (The right tools for the job)

[mjr () nfr net (Marcus J. Ranum)]

> - Cisco is supposedly trying to get their CBAC/IDS stuff into Cat5500/6000
> builds mid next year.  Who knows if it will happen, but its a cool
> thought.  The current build of IOS that has the ID sigs (12.0.5T on some
> platforms) has about 60 signatures.

Will that version do TCP reassembly and sequence tracking? I have
wondered if someone would ever try to push that into a router. My
guess is that the RAM and CPU requirements of doing so will be too
much for a typical router - state tracking on a busy FDDI can take
many megs of buffer space when the network gets congested and
sessions start retransmits.

> - The ODS stuff that I've seen (again, haven't gotten a chance to pound on
> it yet - stay tuned) uses a multiple-segment "backplane."  Essentially,
> its not really embedded in the switch, it just has an inspection unit
> (read: machine with many NICs) attached to each "plane."  I wish someone
> from ODS was on this list....lemme see if I can get some more accurate
> info on this.  Point being though, from what I saw, it was NOT *in* the
> switch.

We were able to boot an NFR appliance CDROM on one of the ODS
devices. That was a huge surprise for us, since it was a version
that only would listen for an Intel Etherexpress Pro 10/100 card
on a generic Intel motherboard. It turns out that the "inspection
unit" is a PC with a NIC and there's a very short wire connecting
it to a span port inside. I'd be surprised if they don't do something
better in a future version. The problem, however, is similar to the
one I alluded to in the router case above - if you put the IDS
logic into your main data path you've got to have a lot of extra
RAM and CPU. Or you have to do very rudimentary IDS. So it makes
sense to put the ID logic on a separate card - that way you can
chew up all the cycles and RAM you like without interfering with
the main data path.

> Unfortunately, (boy am I opening myself up to criticism here - yikes)
> in our testing we just turned everything on, and ran with it.  Our only
> other option was to strip them all down to the lowest common denominator,
> and that is NOT what happens in the real world.  Not to mention its unfair
> to the vendors that do have a wide range of sigs....

I think your approach is reasonable. One of the product testing
concepts I am really fond of is the "just hit return" install.
Perform the absolute minimum configuration necessary to test
the product. That way you'll get a feeling for what it does
when it's installed the way many sites will install it. Of course,
this also means that an unscrupulous vendor could play games
with their default configuration that would make it look better
on paper. I can think of lots of ways someone could do that. :(
Benchmarks can be cooked even without having to lift a finger.

> If/when the CVE stuff matures, and if/when vendors get their internal sigs
> lists/DBs compliant, we can one day list out hundreds of sigs and do a
> real apples to apples comparison.  Until then, however, someone is either
> going to a) go insane trying to map ID product sig lists or b) continue
> doing testing that attempts to be as fair as possible, but not ideal.

There's a few other problems - what about anomalous conditions?
NFR's latest set of filters (we've got about 400 checks in 4.0
and another 400 coming in 4.1 real soon now) checks not just for
known problems, but for things that could indicate a known
problem. I'm not sure whether this is "anomaly detection"
or "misuse detection" - it's more like "protocol boundary checks"
subject lines of SMTP messages should probably not be longer
than a certain amount, FTP RETR commands should not (probably) contain
non-ascii characters, or more than MAXPATHNAMELEN bytes, etc, etc.
It's difficult, maybe impossible, to categorize those kinds
of checks other than "something is weird" checks - just like
the kind of "something is weird" alerts you'll get with pure
statistical anomaly detection systems.

> There are just SO many factors....and let's not forget this: ID vendors
> are not perfect.  Not all of the sigs that we tested worked right.  Some
> didn't work at all.  We found the problems and they are now fixed, but
> know that these products aren't even 100% accurate at what they do look
> for.  So many things to consider......

I don't think any of the vendors claim their products are perfect.
What I think we're all finding is that this, like most security
technologies, is under a period of rapid evolution. Things will
continue to get better, as we all see what works, and steal ideas
from eachother. :) Kinda reminds me of the early days of the firewall
biz: lots of cross-pollination, a few dead ends, and a lot of
innovation.

mjr. (being a trend setter is a b**ch)

--
Marcus J. Ranum, CEO, Network Flight Recorder, Inc.
work - http://www.nfr.net
home - http://www.clark.net/pub/mjr