Re: Feedback on rule testing

[Rob MacGregor <rob.macgregor () gmail com>]

On Fri, Dec 20, 2013 at 5:12 PM, James Dickenson <jdickenson () gmail com> wrote:
> Hey snort users,
>
> I've been talking with some co-workers recently about our in house rule
> development and about ways we could possibly improve it.  I was wondering if
> any of you on the snort user list could give us your experience in regards
> to the process of creating rule you use at where you work or that you submit
> to ET or VRT.  How do you sanity check the rules before you push them to
> your sensors?  Do you have a formal lifecycle process and what does that
> entail?  Do you automate the process somewhat with scripting or software and
> if so how?
>
> Your suggestions and comments are much appreciated,

We run things through 3 automatic steps before we deploy them:

1) Syntax checking (dumbpig and similar)
2) Run through snort with -T to ensure it compiles
3) Deploy to a testing sensor (with live traffic) for 5 minutes and
check the volume of alerts - anything above a defined volume is
automatically rejected and whatever happens the submitter is provided
the flows that hit if any did (this can be over-ridden by an admin if
it turns out they're all true positives and our network is hosed)

We're looking at the option of providing a pcap of known malicious
traffic to confirm the signature fires on the traffic - haven't got
there yet though.

After a signature has deployed we track the true/false positive ratio
(according to the analyst interface), anything above a certain FP
ratio or volume gets flagged automatically for attention, there are
other limits for simply removing the signature. Every 6 months they
have to be reviewed to confirm they should remain deployed (ok,
there's an assumption it's actually reviewed and that the author
hasn't just claimed they have) - that's still a manual process though.

This has, overall, kept our in house signatures to a fairly high
standard. There are still issues, but mandatory training, having
experienced staff check other's signatures and using the ban-hammer on
repeat offenders means that those are minimised these days. Nobody
wants to be the one person in the team who isn't allowed to write
signatures ;)

-- 
                 Please keep list traffic on the list.

Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT 
organizations don't have a clear picture of how application performance 
affects their revenue. With AppDynamics, you get 100% visibility into your 
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!