Re: Assessment tools/Scanners

[dugsong () monkey org (Dug Song)]

On Fri, 8 Oct 1999, Ryan M. Ferris wrote:

> A comprehensive security assessment tool specifically to compare IDS
> is generally not available although Anzen is pushing something like
> this.

what we're pushing for is a better understanding of the failure modes
these products have, and the development of better quality metrics to be
able to quantitatively characterize these systems. see

        http://www.monkey.org/~dugsong/talks/ids/

for an "Intrusion Detection 101" presentation that identifies several
specific and overarching concerns.

for a good qualitative IDS comparison, see Kathleen Jackson's commercial
IDS evaluation matrix for Los Alamos Nat'l Lab, a snippet of which is
available at

        http://www.anzen.com/news/anzen_chart.pdf

> All IDS products are developed with internal test suites that determine
> whether or not a given IDS signature works.

but the problem is that vendors aren't using or developing test suites
that determine how their systems FAIL. this is evident from the ways we've
found to trivially elude them.

-d.