Re: IDS Testing tool

[ADT <synfinatic () gmail com>]

On Mon, 14 Jun 2004 10:59:01 -0400 (EDT), Anton Chuvakin
<anton () chuvakin org> wrote:
> > > What's wrong with just blasting it with a vuln scanner? Nessus will
> > > generate a lot of noise in most NIDSs and can even be tweaked for more
> > > "noisyness"
>
> > Well think about it... a good IDS which limits the number of false
> > positives should detect the actual exploit.  A vulnerability scanner
> > is supposed to check for the vulnerability, *not* to run the actual
> > exploit, b/c then it may crash/root/etc your own box.  Hence, an
> Well, that sounds very sensible in theory. How about you try to scan
> something in view of pretty much any major commercial NIDS - you would get
> tons of alarms staring from 'portscan detected' to all sorts of exploits
> and 'invalid this or that'.

Well, I'd argue that if you ran a portscan, then obviously that's not
a false positive.  And it's likely that many if not all of the
"invalid this or that" are also correct.  But if you see any "exploit
detected" then I'd start asking myself just how accurate that
signature is.  While I'm sure *some* vulnerability scans are almost
identical to the exploit, in my experiance, vulnerability scanners try
to avoid that as much as possible.

> > using Nessus or other vulnerability scanners are a crappy way of
> > testing an IDS.  (Of course if you've got a crappy IDS, then perhaps a
> That depend what you mean by 'testing'. Obviously, if you want to test
> detection quality, blasting with a scanner makes no sense. But, if you are
> testing the alerting channel or testing whether NIDS even sees your
> traffic, there is nothing wrong with Nessus, IMHO.

Well I can't really argue with that... but the original question led
me to believe he was looking to test the accuracy of the IDS.  If they
just want to test logging/alert or wether or not it even sees traffic,
there are probably far easier ways to go about it then installing and
running Nessus or other  vulnerability scanner (nmap would do the job
just fine and is far easier to compile and a lot faster to run for
example).

Basically, there's a reason Nessus and simular tools call themselves
"vulnerability scanners" and not "IDS testers".  If they were good at
testing IDS's, don't you think the authors and/or marketing
departments for these products would say so?

I'm not saying that an IDS should not or will not trigger when a
vulnerability scanner is run, just that the good IDS's shouldn't
detect a majority of the scans as an attack.

-Aaron

---------------------------------------------------------------------------

---------------------------------------------------------------------------