Anomaly detection [was Re: Assessment tools/Scanners]

[stuart () SiliconDefense com (Stuart Staniford-Chen)]

Dug Song wrote:

> >         With a rule-based system, how does one go beyond the list of known
> > attacks to alarm vulnerabilities as well as known threats?
>
> this is one of the failings of misuse detection, and why anomaly detection
> is so important.

I'm not sure that anomaly detection is all that great an idea to install on
an end-system for practical real-world intrusion detection.

A statistical anomaly detection system (which I assume is what you're talking
about) attempts to form a statistical profile of what "normal" activity on
the system looks like, and then checks ongoing activity against the normal
activity to see if it looks too weird.  If it is weird, the system flags it
as intrusive.  This idea was first introduced by Dorothy Denning in her 1986
IDS paper.

A variety of research efforts have built systems which attempted to do this. 
The general difficulties are 
        * the systems sometimes take an unreasonably long time to train.
        * what is statistically "normal" shifts over time (often suddenly - you get
a 
          new application, install a new network, etc).  The system needs to retrain 
          itself when this happens.
        * most things that are abnormal are not intrusive (screwy things happen on 
          networks and computers all the time, and the great bulk of them aren't 
          intrusions).  Statistical systems often can't distinguish.
        * the systems often can only say "this connection is weird" (or the
equivalent),      but not provide much meaningful information about why it's
weird, or what to 
          do about it.

Simplifying the issue grossly, if a new kind of attack shows up around the
world is it better if

        A every IDS has a whole bunch of complex, expensive, logical and statistical 
          algorithms designed to figure out new attacks for themselves

        *or* 

        B all the complex logic and statistics lives, along with a lot 
          of human expertise, in central R&D labs.  Those labs figure out the 
          new attacks and distill them to signatures.  The IDS sensors are dumb 
          things which just download the new signatures as necessary.

I think B is clearly a more practical alternative.  If IDSs do any learning,
it should be very simple and signature-specific learning, not general and
complex learning.

There is another approach besides misuse and anomaly detection, which is
specification-based intrusion detection (which was developed by Calvin Ko at
UC Davis).  The idea is that for some cases, a human can write down a precise
machine-checkable specification for what a program/network service/whatever
should look like in normal use.  Any behaviour outside of this specification
is abnormal and should be flagged.  This is similar to anomaly detection in
that it's "anything that isn't known to be ok is bad", as opposed to
misuse/signature detection "anything which looks like known cases of misuse
is bad".  But it's based on human written specifications rather than
statistical models generated from real data.

This is promising if the thing being specified has very predictable behaviour
normally. Eg, you can write a specification for "what files does the finger
program on my system normally open" and have some hope of success.  However,
you can't write very useful specifications for things humans choose to do (eg
what commands you run at a Unix shell, what web sites you visit).

Stuart.


-- 
Stuart Staniford-Chen --- President --- Silicon Defense
                   stuart () silicondefense com
(707) 822-4588                     (707) 826-7571 (FAX)