RE: False Positives

["Fergus Brooks" <fergusb () evolve-online com>]

10 cents:

There isn't an IDS system that will not report "false positives"
regardless of the definition. While selling IDS systems we use tools
like the excellent IDS Informer to prove the concept. Being as these
tools are not actually attacking but testing, and they report an attack,
then they are generating a *false positive*

I believe this thread was about eliminating them, whatever they are, and
this is the crucial thing that my customers ask me and also my key
interest when looking at products to recommend. 

Any IDS system that issues a huge amount of unimportant alerts becomes a
larger danger to an organisation than no IDS at all. This is because the
business will say they have an IDS however the alerts (good, bad,
irrelevant) are not monitored because 99 times out of 100 (being
generous...) they are rubbish.

In our region (Asia) I would go as far as to say that 95% of the IDS
systems that are implemented are not delivering.

Event correlation is a start - baselining, correlation, and consistent
tweaking and analysis are, IMHO, the best way to eliminate *false
positives*

Always looking for better ways though?



-----Original Message-----
From: Steven Richards [mailto:srichards () netscreen com] 
Sent: Thursday, 5 June 2003 4:19 AM
To: focus-ids () securityfocus com
Cc: 'Harshul Nayak (ealcatraz)'; Andi Hess
Subject: RE: False Positives


 
-----BEGIN PGP SIGNED MESSAGE-----

<insert standard vendor disclaimer here>

I would like to offer up a couple of definitions of terms for
discussion.

We in the security space, more specifically the IDS/P space should agree
on some standard language.


In my opinion, these terms should be considered:

False Positive=   Sensor is supposed to be looking for 'xyz' and what
actually goes across the wire is 'abc' which generates an alert.

Non-Security Event=   Sensor is looking for 'XXX' it sees 'XXX' go
over the wire, *but* it is not an actual "Security Event" because the
corporate security policy and/or the system configurations are
intentionally not configured to disallow 'XXX'.  For example: you
configure your systems to allow Null Login NetBIOS Sessions (for
whatever reason) and your corporate security policies do not disallow
this.  The sensor sees this traffic and it generates an alert.  It
actually happened on your network.  It's just that you don't *care*
about it.




> -----Original Message-----
> From: Harshul Nayak (ealcatraz) [mailto:harshul () ealcatraz com]
> Sent: Wednesday, June 04, 2003 1:11 PM
> To: Andi Hess
> Cc: focus-ids () securityfocus com
> Subject: RE: False Positives
>
>
> Hello Andi,
>
> It's quite often many people use the term "false positive"
> for the tests
> conducted by you,
> With due respect to all, would like to share with reference 
> to the article
> by Marcus J. Ranum for ICSA Labs IDSC :
>
> Your mail is referring to :
>
> * False Attack Stimulus - A stimulus that causes an IDS to
> trigger an alarm
> when no actual exploited attack has
> occurred. False attack stimuli generate false positives; and 
> are frequently
> seen during badly designed IDS tests or
> when attackers attempt to overload an IDS' alert processing 
> capability using
> a tool such as Stick. Many scanning
> tools generate false attack stimuli. For example, if a
> vulnerability assessment tool connects to a web server and
> issues a "GET" for a known-vulnerable CGI-bin script, it is 
> not the same
> thing as when a hacking tool connects and
> exercises the complete attack via the same script.
> Depending on the application protocols in use it may be 
> difficult for the
> IDS to distinguish a stimulus that looks for a
> vulnerability from a stimulus that actually triggers a 
> compromise in the
> system. False attack stimuli are deliberately
> used in some IDS testing regimens, attempting to verify the 
> IDS' function
> without placing real systems at risk. When
> testing IDS a tester should mix a number of false attack 
> stimuli with true
> attack stimuli.
>
> Here is the definition of "false positive"
> * False Positive - An alarm generated by an IDS in which the
> IDS alerts to a
> condition that is actually benign. In
> other words, the IDS made a mistake. A typical example of a 
> false positive
> would be a case when an IDS raises a
> "SYN flood" alarm because it sees a large number of SYN 
> packets directed at
> a busy web server and mistakenly
> concludes it is under attack. Another example of a false 
> positive would be
> an IDS raising a "SMTP Wiz attack" alarm
> when it observes the string "DEBUG" in the body of an SMTP message.
>
> hope this helps you.
>
> -regs
> Harshul
> Copyright C 2002 Sintelli
> http://www.sintelli.com
> ----------------------------------------------------------------
> "A good listener is usually thinking about something else."
>
> -----Original Message-----
> From: Andi Hess [mailto:andi_hess () web de]
> Sent: Tuesday, June 03, 2003 4:13 PM
> To: focus-ids () securityfocus com
> Subject: False Positives
>
>
>
>
> Hi there,
>
> I am new in the field of NIDS and I wonder if the
> problem of false positives is really this huge as
> mentioned in several publications.
>
> I am considering tools like PCP, Stick (I have never
> seen them, but read about them) which can be used to
> generate huge amount of packets and each on triggers an
> alarm on the victim IDS (a false positive, as the
> packets are not a real attack).
> As it has been impossible for me to find any of the
> above mentioned packet generators - I wonder how the
> packets look like?
> Is it possible to differentiate 'artifically' generated
> false positives from natural ones?
>
> Any hint is welcome!
>
> Thank you.
>
> A.
>
>
>
>
> --------------------------------------------------------------
> --------------
> ---
> INTRUSION PREVENTION: READY FOR PRIME TIME?
>
> IntruShield now offers unprecedented Intrusion IntelligenceTM
> capabilities
> - including intrusion identification, relevancy, direction, impact
> and analysis
> - enabling a path to prevention.
>
> Download the latest white paper "Intrusion Prevention: Myths,
> Challenges,
> and Requirements" at:
> http://www.securityfocus.com/IntruVert-focus-ids2
> --------------------------------------------------------------
> --------------
> ---
>
>
> --------------------------------------------------------------
> -----------------
> INTRUSION PREVENTION: READY FOR PRIME TIME?
>
> IntruShield now offers unprecedented Intrusion IntelligenceTM
> capabilities 
> - including intrusion identification, relevancy, direction, 
> impact and analysis 
> - enabling a path to prevention.
>
> Download the latest white paper "Intrusion Prevention: Myths,
> Challenges, and Requirements" at: 
> http://www.securityfocus.com/IntruVert-focus-ids2
> --------------------------------------------------------------
> -----------------

-----BEGIN PGP SIGNATURE-----
Version: PGP 7.0

iQEVAwUBPt5VYWNGcU7aCIRPAQG9+ggAgnFVJUYL/AL65LrpGGs54SKtbX2uze49
xOneZcytR5GDi0PROh139SMB6ytrleIQjodtckAxGsZxsH4nASL2XG88FfjyXOkE
l8gj6QyQoQGCE2FQf8333q1FOkCtT2BSm0YpY1/ekVgeDhZz05iuG++hF5IUPQZg
l3YMv2Oes1/mW9jOS3MOP4Bggq7xnKb7LBPr7eJFmhW0HZe+AR8knj2aV54OB3+N
DQhfgrBDKAM+Xqg/cqaG5L5KVejfRue7JEWfTt7kqDBP8hdxejAQJx3G7sb1yj5/
gN2Kq4SKBP0f1hbBPxmRC2+oEGffy/+hAW4DiMg/NhkZb/IvvPMAqw==
=wkY4
-----END PGP SIGNATURE-----

------------------------------------------------------------------------
-------
INTRUSION PREVENTION: READY FOR PRIME TIME?

IntruShield now offers unprecedented Intrusion IntelligenceTM
capabilities 
- including intrusion identification, relevancy, direction, impact and
analysis 
- enabling a path to prevention.

Download the latest white paper "Intrusion Prevention: Myths,
Challenges, and Requirements" at: 
http://www.securityfocus.com/IntruVert-focus-ids2
------------------------------------------------------------------------
-------

--
This message has been scanned by AVMail.



-------------------------------------------------------------------------------
INTRUSION PREVENTION: READY FOR PRIME TIME?

IntruShield now offers unprecedented Intrusion IntelligenceTM capabilities 
- including intrusion identification, relevancy, direction, impact and analysis 
- enabling a path to prevention.

Download the latest white paper "Intrusion Prevention: Myths, Challenges, and Requirements" at: 
http://www.securityfocus.com/IntruVert-focus-ids2
-------------------------------------------------------------------------------