Re: IDS Testing (WAS: Re: kernel implementations)

[Dug Song <dugsong () monkey org>]

Archive: http://msgs.securepoint.com/ids
FAQ: http://www.ticm.com/kb/faq/idsfaq.html
IDS: http://www-rnks.informatik.tu-cottbus.de/~sobirey/ids.html
HELP: Having problems... email questions to ids-owner () uow edu au
NOTE: Remove this section from reply msgs otherwise the msg will bounce.
SPAM: DO NOT send unsolicted mail to this list.
UNSUBSCRIBE: email "unsubscribe ids" to majordomo () uow edu au
-----------------------------------------------------------------------------
On Sun, 23 Jul 2000, Greg Shipley wrote:

> First off, the Los Alomos paper referenced here:
> http://www.anzen.com/news/anzen_chart.pdf
> I don't get it.  This is a thorough chart of features - great.  Is there
> *ANY* testing behind this report, what-so-ever?  ANY?

no, but this wasn't really the point of the report. you can read it in
full at

        http://lib-www.lanl.gov/la-pubs/00416750.pdf

> Second, the paper referenced here:
> http://www.zurich.ibm.com/pub/Other/RAID/Prog_RAID98/Program.html -
> from Roy Maxion at RAID98... While I respect the author's time and
> effort, and there are some good points made, I still fail to see how
> this is ANYWHERE CLOSE to providing a framework for IDS testing...or
> even CLOSE to addressing the issues at hand.

Maxion attacks the meta-problem of developing IDS test methodology,
instead of developing discrete IDS tests. check out his recent paper on
"Benchmarking Anomaly-Based Detection Systems" for an application of this
to a specific case, though:

        http://www.cs.cmu.edu/afs/cs.cmu.edu/user/maxion/www/pubs/maxiontan00.pdf

> Since this list is populated by all of the big players, and everyone
> insists that there is really poor testing and comparisons going on
> these days, why not hammer those issues out here?  Why not come up
> with a framework for testing and comparisons?

it certainly hasn't been for want of trying, i'll say that much.

i'll respond to your specific IDS test criteria later - i rode the world's
tallest, fastest rollercoaster today (along with many lesser ones) and i
need some time to recuperate... :-/

-d.

---
http://www.monkey.org/~dugsong/