Anyones doomsday machine running low on IDS analyst tears?

[Will Metcalf <william.metcalf () gmail com>]

I kid... I kid.. But seriously ran into something sort of interesting
playing snort conf options.  It seems that the default pm in snort has
changed the the default pattern matcher to be ac-split via...

# Configure the detection engine  See the Snort Manual, Configuring
Snort - Includes - Config
config detection: search-method ac-split search-optimize max-pattern-len 20

If you are using your own custom rule sets/ET rules this means that if
you previously had an override for fast_pattern's default cut-off of
20 via offset and length options via fast_pattern:<offset>,<length>;
or you expected the entire unique pattern specified via fast_pattern,
You are now limited to only 20 bytes for fast_pattern even if you set
offset/length. So for example let us consider the following ET
signature optimized for 2.8.6 using the default pm in 2.8.6 of
ac-bnfa.

alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"ET TROJAN
Delf Checkin via HTTP (8)"; flow:established,to_server;
content:"POST"; http_method; content:".php"; http_uri; nocase;
content:"User-Agent|3a| Mozilla/3.0 (compatible|3b| Indy Library)";
http_header; fast_pattern; content:"name="; http_client_body; depth:5;
classtype:trojan-activity;
reference:url,doc.emergingthreats.net/2008268;
reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/VIRUS/TROJAN_Delf;
sid:2008268; rev:5;)

  Fast pattern matcher: HTTP Header content
  Fast pattern set: yes
  Fast pattern only: no
  Negated: no
  Pattern offset,length: none
  Pattern truncated: no
  Original pattern
    "User-Agent:|20|Mozilla/3.0|20|(compatible|3B 20|Indy|20|Library)"
  Final pattern
    "User-Agent:|20|Mozilla/3.0|20|(compatible|3B 20|Indy|20|Library)"

timestamp: 1286391049
Rule Profile Statistics (all rules)
==========================================================
   Num      SID GID Rev     Checks   Matches    Alerts
Microsecs  Avg/Check  Avg/Match Avg/Nonmatch
   ===      === === ===     ======   =======    ======
=========  =========  ========= ============
     1  2008268   1   5         18         4         4
         55        3.1        3.8          2.9

timestamp: 1286391054
Rule Profile Statistics (all rules)
==========================================================
   Num      SID GID Rev     Checks   Matches    Alerts
Microsecs  Avg/Check  Avg/Match Avg/Nonmatch
   ===      === === ===     ======   =======    ======
=========  =========  ========= ============
     1  2008268   1   5         18         4         4
          52        2.9        3.8          2.7

timestamp: 1286391058
Rule Profile Statistics (all rules)
==========================================================
   Num      SID GID Rev     Checks   Matches    Alerts
Microsecs  Avg/Check  Avg/Match Avg/Nonmatch
   ===      === === ===     ======   =======    ======
=========  =========  ========= ============
     1  2008268   1   5         18         4         4
        54        3.0        3.9          2.8


Ok this is what I expect. I have a long match that should be fairly unique.

Lets modify the pm to be the one included in the VRT version of the
default snort.conf and/or the 2.9.0 snort.conf.  We now get
"User-Agent:|20|Mozilla/" as the match added to fast_pattern.  Uh
oh... This isn't a very unique pattern is it, this will cause all
packets that contain a firefox UA to be at least partially evaluated.

1:2008268
  Fast pattern matcher: HTTP Header content
  Fast pattern set: yes
  Fast pattern only: no
  Negated: no
  Pattern offset,length: none
  Pattern truncated: 50 to 20 bytes
  Original pattern
    "User-Agent:|20|Mozilla/3.0|20|(compatible|3B 20|Indy|20|Library)"
  Final pattern
    "User-Agent:|20|Mozilla/"

timestamp: 1286390771
Rule Profile Statistics (all rules)
==========================================================
   Num      SID GID Rev     Checks   Matches    Alerts
Microsecs  Avg/Check  Avg/Match Avg/Nonmatch
   ===      === === ===     ======   =======    ======
=========  =========  ========= ============
     1  2008268   1   5      17381         4         4
10212        0.6        4.7          0.6

timestamp: 1286390775
Rule Profile Statistics (all rules)
==========================================================
   Num      SID GID Rev     Checks   Matches    Alerts
Microsecs  Avg/Check  Avg/Match Avg/Nonmatch
   ===      === === ===     ======   =======    ======
=========  =========  ========= ============
     1  2008268   1   5      17381         4         4
9516        0.5        4.7          0.5

timestamp: 1286390779
Rule Profile Statistics (all rules)
==========================================================
   Num      SID GID Rev     Checks   Matches    Alerts
Microsecs  Avg/Check  Avg/Match Avg/Nonmatch
   ===      === === ===     ======   =======    ======
=========  =========  ========= ============
     1  2008268   1   5      17381         4         4
9347        0.5        4.5          0.5

timestamp: 1286390784
Rule Profile Statistics (all rules)
==========================================================
   Num      SID GID Rev     Checks   Matches    Alerts
Microsecs  Avg/Check  Avg/Match Avg/Nonmatch
   ===      === === ===     ======   =======    ======
=========  =========  ========= ============
     1  2008268   1   5      17381         4         4
9266        0.5        4.1          0.5

Evaluating the same pcap we now spend 20x more ticks evaluating the
same rule.  I doubt that specifying a fast pattern offset and length
will allow us to override the max-pattern-len 20 but lets give it a
shot by specifying 0,50 as the offset and length.

snaplen = 65535
1:2008268
  Fast pattern matcher: HTTP Header content
  Fast pattern set: yes
  Fast pattern only: no
  Negated: no
  Pattern <offset,length>: 0,50
    "User-Agent:|20|Mozilla/3.0|20|(compatible|3B 20|Indy|20|Library)"
  Pattern truncated: 50 to 20 bytes
  Original pattern
    "User-Agent:|20|Mozilla/3.0|20|(compatible|3B 20|Indy|20|Library)"
  Final pattern
    "User-Agent:|20|Mozilla/"

No dice.. So I guess the take away here is that if you are moving to a
VRT snort.conf or a 2.9.0 ruleset and you are running custom rules I
would pay real close attention to debug-print-fast-pattern output.  We
are going through the poor performers now and making modifications
where appropriate for ET rules, just thought folks might want to know
;-)...

Regards,

Will

------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users