IDS: Re: IDS testing methodologies

[Ron Gula <rgula () tenablesecurity com>]

At 08:42 PM 12/30/2003 +0100, Henrik Falkenthros, direktoer wrote:
> Hi List !
>
> I'm trying to find out ways of testing different IDS systems; is there a
> 'recommended'/best practise methodology for testing Network based IDS (NIDS)
> ? Any information - papers, tools, links and own experience are much
> appreciated,,, 8-)
>
> cheers, Henrik Falkenthros
>
>
> ---------------------------------------------------------------------------
> ---------------------------------------------------------------------------

When I was running Dragon IDS development, we'd get ask to help
potential customers with their 'testing' of an IDS. I used to
see folks test 5 different NIDS, with 100s of different parameters.
It was usually useless because the development cycle of most of
these NIDS was less than the decision cycle of most large
enterprises. Nowadays I tell people to do a paper study, get some
reference accounts you can talk to, choose two solutions and go
right to a pilot deployment.

What you use to test depends more on what you want out of the
vendor or solution.

Here are things I would recommend that you need to test when
looking at an IDS:

- the baseline security of the installed devices and their
  management systems.

- the performance of the underlying data-store/data-base after
  it has been running for 1-2 moths

- how does it handle *your* live traffic. If you can't deploy
  it on your network, get a sniffer, collect the data, bring
  it back to the lab and replay it.

- frequency/accuracy of signature updates

- spend some time up front to see if your vendors can actually
  sell to your organization. I've heard to many stories where
  certain products were selected and shot down because of the
  wrong VC backer, alumni, contract, country, .etc.

This sounds really bad, but spending time on actually seeing
if a NIDS is actually catching intrusions and trying to find
ways to bypass it is not the best use of your time. A lot of
other people have already done this and regularly publish their
results.

Ron Gula, CTO
Tenable Network Security
http://www.tenablesecurity.com





















  

---------------------------------------------------------------------------
---------------------------------------------------------------------------