Re: Announcement: Alert Verification for Snort

[Michael Krieger <michael.krieger () linznet at>]

Hi everybody,

	I spent the last few months developing and improving QuIDScor (an 
IDS-VA-correlation engine - http://quidscor.sourceforge.net) and spent 
also some time thinking about the different cases and the terminology 
which should be used. For the term "false positive" I agree with Martin 
Roesch, that alerts for attacks which have no impact can not be 
classified as "false positives" in the context of intrusion detection.

On Donnerstag, Oktober 23, 2003, at 12:35  Uhr, Raistlin wrote:

> Actually, I think that vulnerable or non-vulnerable is not tied to the
> true/false positive concept... so I'd say:

I totally agree, that from an IDS point of view vulnerable / 
non-vulnerable is not tied to the true/false positive concept and that 
there is a difference between not detected attacks with present 
signatures and not detected attacks because of not present signatures. 
But again you get six cases if you add the existence of "signature 
present" (for signature based systems).

I personally would differentiate the following six cases:

1) attack present, signature present, detect - correct detection
2) no attack, signature present, detect - false positive
3) attack present, no signature, no detect
4) no attack, no signature, no detect
5) attack present, signature present, no detect - false negative
6) no attack, signature present, no detect - true negative

Personally I also think that the term "alert verification" is not used 
correctly. As current IDS are often "just" used as attack detectors 
(more like attack detection scanners than intrusion detection scanners) 
and start to become IDS with the help of correlation with vulnerability 
scans, it is more an "attack validation" than an "alert verification". 
A vulnerability scan is not able to tell whether the alert was correct, 
as it has no information about the actual traffic. I would even 
consider redefining the name of the technology currently called IDS to 
attack / event detection scanner / system (ADS / EDS) and define the 
correlation of those ADS /EDS with VA-data as "intrusion detection".


As Ron Gula mentioned his 9 cases of IDS-VA-correlation, there are 
actually many more. :-)

Just for everybody to explain all possible IDS-VA-correlation cases:

Like the six cases mentioned above for IDS, for (signature-based) 
vulnerability scanners exist following cases:

1) vulnerability present, signature present, detect - correct detection
2) no vulnerability, signature present, detect - false positive
3) vulnerability present, no signature, no detect
4) no vulnerability, no signature, no detect
5) vulnerability present, signature present, no detect - false negative
6) no vulnerability, signature present, no detect - true negative

If you now try to correlate IDS with VA-scans you could actually get 
(at least in theory) 36 different cases (6 cases of IDS multiplied by 6 
cases of VA-scans).

Well, that's at least my point of view,

        Michael


---------------------------------------------------------------------------
Network with over 10,000 of the brightest minds in information security
at the largest, most highly-anticipated industry event of the year.
Don't miss RSA Conference 2004! Choose from over 200 class sessions and
see demos from more than 250 industry vendors. If your job touches
security, you need to be here. Learn more or register at
http://www.securityfocus.com/sponsor/RSA_focus-ids_031023 
and use priority code SF4.
---------------------------------------------------------------------------