Re: Reliability of signatures

[Matthew Jonkman <jonkman () emergingthreatspro com>]

I agree on the difference between just logging hits and having true FP and TP ratings. But even a false positive can be 
different on the same packet in different organizations. Many folks mark a hit a false positive because it's just not 
of interest, vs nt hitting on what it's supposed to be looking for. 

I don't see real good ways to make that distinction en mass, I certainly wouldn't want to have to mark events that way 
in addition to the usual handling of events. 

I think there is definitely value in just tracking raw hits. Few things off the top of my head:

1. A new sig is out and we get massive numbers of hits, more than should be expected (ie hits per site ratio). That 
should be a red alert for a bad sig.

2. Once a sig is established for a few weeks and is stable, then any fluctuation is significant. Especially malware 
sigs and bot stuff. New strain, new outbreak, old strain using a new o-day, etc. 

3. Established malware/bot sig, suddenly drops to zero after a period of hits. We're being evaded and it needs 
attention asap (unless the botnet was infiltrated and killed)

4. Geo location of sources would also be extremely interesting at scale.

These were some of the things I wanted to do with sidreporter, but never had the resources to pursue. I'm sure there 
are many more things we could infer just from raw hit patterns.

Matt


On Feb 4, 2011, at 12:56 PM, Martin Holste wrote:
> Ok, cool.
>
> So, here's my feedback to SF/ET regarding what will help, and I'll try
> to summarize the above comments to be sure I have understood them:
>
> 1. Up/down vote per gid:sid:rev my analysts can click on at the tail
> end of an investigation to indicate that something's been helpful with
> a way to make a note of how it was helpful.
> 2. Dshield/sidreporter-style automated submissions so that you guys
> can see the sigs that are flagging on all kinds of FP's right off the
> bat and also to get a cross-section of what IP's are flagging alerts.
> 3. Up/down vote for category confidence on a given gid:sid:rev.
> And, I'd personally add a fourth that I feel is very important:
> 4. Tag suggestion for a gid:sid:rev with corresponding up/down vote
> for confidence.
>
> I personally want to see 1 and 4 implemented ASAP, and they can be
> started without retrofitting to all existing signatures.  Each datum
> contributed is value added.
>
> ------------------------------------------------------------------------------
> The modern datacenter depends on network connectivity to access resources
> and provide services. The best practices for maximizing a physical server's
> connectivity to a physical network are well understood - see how these
> rules translate into the virtual world? 
> http://p.sf.net/sfu/oracle-sfdevnlfb
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users


----------------------------------------------------
Matthew Jonkman
Emergingthreats.net
Emerging Threats Pro
Open Information Security Foundation (OISF)
Phone 765-807-8630
Fax 312-264-0205
http://www.emergingthreatspro.com
http://www.openinfosecfoundation.org
----------------------------------------------------

PGP: http://www.jonkmans.com/mattjonkman.asc




------------------------------------------------------------------------------
The modern datacenter depends on network connectivity to access resources
and provide services. The best practices for maximizing a physical server's
connectivity to a physical network are well understood - see how these
rules translate into the virtual world? 
http://p.sf.net/sfu/oracle-sfdevnlfb
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users