RE: RE: Determining when something is NOT random

[Max Kilger <MaxK () symresources com>]

Archive: http://msgs.securepoint.com/ids
FAQ: http://www.ticm.com/kb/faq/idsfaq.html
IDS: http://www-rnks.informatik.tu-cottbus.de/~sobirey/ids.html
HELP: Having problems... email questions to ids-owner () uow edu au
NOTE: Remove this section from reply msgs otherwise the msg will bounce.
SPAM: DO NOT send unsolicted mail to this list.
UNSUBSCRIBE: email "unsubscribe ids" to majordomo () uow edu au
-----------------------------------------------------------------------------
The chi-square test is indeed one way to provide evidence that the hits you
are receiving from some address category is probably not due to chance.  One
modification that I would make (and perhaps this is implied in their
message) is that you should base the expected cell counts on a priori data
rather than from "random chance".  Take a very simple example:

1)  Assume there are four IP addresses that you are concerned about - call
them W, X, Y and Z
2)  By chance alone you would expected that the expected probability for
each cell would be .25
3)  You know in real life from a priori data that you have collected as a
baseline that
    p(W) = .2
    p(X) = .4
    p(Y) = .1
    p(Z) = .3
4)  Use the a priori probabilities from step 3 to calculate your expected
cell counts rather than .25

That should "tune" your chi-square to your particular situation.  You should
also note that the chi-square test is very sensitive to large sample size.
While this usually drives statisiticans crazy (every chi-aquare test they
attempt on their data set is statistically significant) it actually has a
benefit to IDS because the sensitivity can help you find that attack lost in
the din of the data stream.

Cheers,

Max Kilger, Ph.D.
Director of Statistical Sciences
Symmetrical Resources

"With all the high technology being integrated into our infrastructure we've
run out of smart people
to run the stuff, so we've resorted to using winged monkeys - and we're
starting to run out of winged monkeys..."




-----Original Message-----
From: Bill Royds [mailto:broyds () home com]
Sent: Monday, July 24, 2000 7:49 PM
To: Lance Spitzner; ids () uow edu au
Subject: IDS: RE: Determining when something is NOT random


Archive: http://msgs.securepoint.com/ids
FAQ: http://www.ticm.com/kb/faq/idsfaq.html
IDS: http://www-rnks.informatik.tu-cottbus.de/~sobirey/ids.html
HELP: Having problems... email questions to ids-owner () uow edu au
NOTE: Remove this section from reply msgs otherwise the msg will bounce.
SPAM: DO NOT send unsolicted mail to this list.
UNSUBSCRIBE: email "unsubscribe ids" to majordomo () uow edu au
----------------------------------------------------------------------------
-
The statistical method of determining whether something is random on not is
the Chi-squared test.
You calculate the sum of squares of (expected-observed)/expected for classes
of something.
Here the expected distribution of IP's can be determined by allocation of IP
blocks. You would expect more IP's hitting you from the densely populated
24.x.x.x class A space. You would not expect something from NortelNetworks
internal class A space. 
Classify the source IP's by class A prefix say, then see if they are fairly
evenly distributed over class A space. If they are, then it is most likely
random. If not, then examine to see which groups are more clumped. If they
are the groups with more active IP addresses, it would give evidence of
actual hacked machines.


-----Original Message-----
From: owner-ids () uow edu au [mailto:owner-ids () uow edu au]On Behalf Of
Lance Spitzner
Sent: Monday, July 24, 2000 00:13
To: ids () uow edu au
Subject: IDS: Determining when something is NOT random


Archive: http://msgs.securepoint.com/ids
FAQ: http://www.ticm.com/kb/faq/idsfaq.html
IDS: http://www-rnks.informatik.tu-cottbus.de/~sobirey/ids.html
HELP: Having problems... email questions to ids-owner () uow edu au
NOTE: Remove this section from reply msgs otherwise the msg will bounce.
SPAM: DO NOT send unsolicted mail to this list.
UNSUBSCRIBE: email "unsubscribe ids" to majordomo () uow edu au
----------------------------------------------------------------------------
-
Are there any tools/techniques to determining when something
is NOT random.

For example, I have a system that was hit with ICMP_ECHO 
packets from 47 systems within two hours.  Based on the
packets, I can determine that the same tool was used
to generate them.  What I want to determine is if the 47
source systems were randomly generated by the tool (as 
often done by Syn Flooding tools) or if the 47 systems 
involved were not randomly generated.  If the 47 Src systems 
were NOT randomly gerenerated, this may indicate that all 47
systems are actuall systems used in the 'attack'.

Any tool that can take a list of IP addresses and determine
if they are random or not?

Thanks!

Lance Spitzner
http://www.enteract.com/~lspitz/papers.html