RE: Assessment tools/Scanners

[broyds () home com (Bill Royds)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

- -----Original Message-----
From: owner-ids () uow edu au [mailto:owner-ids () uow edu au]On Behalf Of
Greg Shipley
Sent: Monday, October 11, 1999 06:46
To: Vin McLellan
Cc: Dug Song; Ryan M. Ferris; ids () uow edu au
Subject: Re: IDS: Assessment tools/Scanners

        On Sun, 10 Oct 1999, Vin McLellan wrote:

        >         How does one develop a test suite that will identify the Failure
        > when an IDS module does not identify the threat behind a novel attack?  (Or
        > an old attack presented in a novel manner?)
        *snip*

        Wow - there are about a billion GOOD questions in here - enough to spawn
        off at least a dozen sub-threads.  I'll try to throw some opinions out
        here, maybe some of them will help....maybe not....dunno....

- ----------------------
Given that there are two kinds of Intrusions, Network and Host Intrusions, there are 2 kinds of failures of Intrusion 
Detection.
The first is failing to detect an authorised access to a network (packets not part of legitimate traffic). The second 
is failing to
detect unauthorised usage of a host.
For a network, there is a hierarchy of possible attacks.
        Given packets on a network, are they
1: Invalid format that servers can't properly handle (bad combinations of flags, fragments with too large or too small 
packets etc.)
One can systematically and exhaustively generate combinations of bad flags in test packets. one can also do case 
testing of
overflows, underflows etc.

2: Valid format but for illegitimate services (Trojans, FTP PUT for restricted pages, disallowed access to data etc.). 
This can be
handled by using a network security policy then systematically violating it.

3. Valid Format for valid services but with invalid authorisation. This is the hardest to detect because the IDS must 
have access to
system security databases. A statistical approach might help here as well as signatures.

  Given a process on a host, there are also several kinds of general attack:
1: Illegitimate process (unauthorised user, unauthorised service)
2. Legitimate process but unauthorised data.
3. Legitimate process, legitimate data, unauthorised time or server.

  By looking at this hierarchy, you can categorise test cases and test results. A test suite should include attempts to 
create each
of these anomalies to categorise strengths and weaknesses. Most exploits can be categorised in this way as well so one 
can
extrapolate the IDS ability to detect anomalous behaviour.

        >         With a rule-based system, how does one go beyond the list of known
        > attacks to alarm vulnerabilities as well as known threats?

        Just to clarify terminology - when you say "rules based" you are referring
        to the traditional "attack signature" model, and not something like ODS'
        CMDS, correct?  That being the case, you can "judge" on a number of
        things, but they are HIGHLY dependent on the target audience.  For
        example, some of the things I've been looking at are:

        - depth of signature-base (how much can it look for and spot?)
        - architecture (does it scale?  How does management work?)
        - level of polish (I have yet to come up with a good way of saying this -
                basically, does it have a useful interface, does it supply good documentation, etc.)
        - capabilities (how customisable is it?  Can it do packet re-assembly?)

        Now, those are things I think are important, but that might not hold true
        for everyone.  Take something like Dragon - very easy to customise, and
        still very raw....but it works.  Some people might not care that it
        doesn't have an attack-tree like display, or that the documentation for
        the attacks are in a separate file, etc.  Others may require that the IDS
        snaps into HP OpenView.  While other may want to puke at the thought of
        OpenView.....It all goes back to the question of what you need the IDS to
        do.  But you are right, there are some common areas you can cover....

Another test of rules based IDS is to seed ordinary traffic with exploits to test whether the IDS can separate the 
wheat from the
chaff. Too many false positives can create as much a problem sometimes as missing an intrusion.

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 6.0.2i
Comment: Bill Royds CAST

iQA/AwUBOAIz7ccYG6mh8NzqEQLHxACg5RPQVykii2GjDr6yXncEY04fOEoAoJmJ
w697LWmpTx6B0Mc25qgEpjZY
=sGnJ
-----END PGP SIGNATURE-----