Snort mailing list archives

Previous By Date Next
Previous By Thread Next

Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3

From: "C. L. Martinez" <carlopmart () gmail com>
Date: Fri, 31 May 2013 06:51:31 +0000
On Thu, May 30, 2013 at 12:45 PM, C. L. Martinez <carlopmart () gmail com> wrote:

Hi all,

 According to the following stats:

May 30 11:46:22 nsm01 snort[30096]:
===============================================================================
May 30 11:46:22 nsm01 snort[30096]: Packet Performance Summary:
May 30 11:46:22 nsm01 snort[30096]:    max packet time       : 10000 usecs
May 30 11:46:22 nsm01 snort[30096]:    packet events         : 654
May 30 11:46:22 nsm01 snort[30096]:    avg pkt time          : 27.1384 usecs
May 30 11:46:22 nsm01 snort[30096]: Rule Performance Summary:
May 30 11:46:22 nsm01 snort[30096]:    max rule time         : 4096 usecs
May 30 11:46:22 nsm01 snort[30096]:    rule events           : 20
May 30 11:46:22 nsm01 snort[30096]:    avg rule time         : 1.046 usecs
May 30 11:46:22 nsm01 snort[30096]:
===============================================================================
May 30 11:46:22 nsm01 snort[30096]: Packet I/O Totals:
May 30 11:46:22 nsm01 snort[30096]:    Received:     69971576
May 30 11:46:22 nsm01 snort[30096]:    Analyzed:     22427618 ( 32.052%)
May 30 11:46:22 nsm01 snort[30096]:     Dropped:     41532168 ( 37.247%)
May 30 11:46:22 nsm01 snort[30096]:    Filtered:            0 (  0.000%)
May 30 11:46:22 nsm01 snort[30096]: Outstanding:     47543958 ( 67.948%)
May 30 11:46:22 nsm01 snort[30096]:    Injected:            0
May 30 11:46:22 nsm01 snort[30096]:
===============================================================================
May 30 11:46:22 nsm01 snort[30096]: Breakdown by protocol (includes
rebuilt packets):
May 30 11:46:22 nsm01 snort[30096]:         Eth:     22436767 (100.000%)
May 30 11:46:22 nsm01 snort[30096]:        VLAN:            0 (  0.000%)
May 30 11:46:22 nsm01 snort[30096]:         IP4:     22436767 (100.000%)
May 30 11:46:22 nsm01 snort[30096]:        Frag:           12 (  0.000%)
May 30 11:46:22 nsm01 snort[30096]:        ICMP:       110634 (  0.493%)
May 30 11:46:22 nsm01 snort[30096]:         UDP:       752816 (  3.355%)
May 30 11:46:22 nsm01 snort[30096]:         TCP:     19433478 ( 86.614%)

using snort under OpenBSD 5.3 doesn't returns good performance. Host
is a Intel(R) Xeon(R) CPU E5620 @ 2.40GHz, with 8 GiB RAM and four
e1000 interfaces.

 In this sensor, I only use so_rules:

# dynamic library rules
# include $SO_RULE_PATH/bad-traffic.rules
# include $SO_RULE_PATH/chat.rules
include $SO_RULE_PATH/dos.rules
include $SO_RULE_PATH/exploit.rules
# include $SO_RULE_PATH/icmp.rules
# include $SO_RULE_PATH/imap.rules
include $SO_RULE_PATH/misc.rules
include $SO_RULE_PATH/multimedia.rules
include $SO_RULE_PATH/netbios.rules
# include $SO_RULE_PATH/nntp.rules
include $SO_RULE_PATH/p2p.rules
include $SO_RULE_PATH/smtp.rules
# include $SO_RULE_PATH/snmp.rules
include $SO_RULE_PATH/specific-threats.rules
include $SO_RULE_PATH/web-activex.rules
include $SO_RULE_PATH/web-client.rules
include $SO_RULE_PATH/web-iis.rules
include $SO_RULE_PATH/web-misc.rules

and monitored network is a 1GiB network.

 Any ideas why??



More info:


top:
load averages:  0.69,  0.65,  0.53
31 processes: 30 idle, 1 on processor
CPU0 states:  2.8% user,  0.0% nice,  0.4% system, 20.4% interrupt, 76.4% idle
CPU1 states:  2.2% user,  0.0% nice,  0.8% system,  0.0% interrupt, 97.0% idle
CPU2 states:  3.0% user,  0.0% nice,  3.4% system,  0.0% interrupt, 93.6% idle
CPU3 states:  6.0% user,  0.0% nice,  5.0% system,  0.0% interrupt, 89.0% idle
Memory: Real: 587M/2947M act/tot Free: 5012M Cache: 2213M Swap: 0K/6142M

  PID USERNAME PRI NICE  SIZE   RES STATE     WAIT      TIME    CPU COMMAND
14655 root       4    0  393M  183M sleep/1   bpf       8:44 14.26% snort
25669 root       4    0 1132K 1740K sleep/2   bpf       0:06  3.52% daemonlogger

systat ifstat (snort process is listening in em3)

    3 users    Load 0.89 0.71 0.56                     Fri May 31 06:23:13 2013

IFACE            STATE  DESC
                     IPKTS   IBYTES    IERRS    OPKTS   OBYTES
OERRS    COLLS
em0              up
                         2      132        0        0      261
0        0
em1              up
                         0      126        0        0      131
0        0
em2              up
                     10348  3425952        0        0        0
0        0
em3              up
                     10346  3425044        0        0        0
0        0


systat mbufs


IFACE             LIVELOCKS  SIZE ALIVE   LWM   HWM   CWM
System                    0   256   185          56
                               2k   171         435
lo0
em0                            2k     6     4   256     6
em1                            2k     6     4   256     4
em2                            2k    66     4   256    66
em3                            2k    65     4   256    65


Stats with ALL so_rules disabled (5 min, more or less):

Rule application order:
activation->dynamic->pass->drop->sdrop->reject->alert->log
Verifying Preprocessor Configurations!
ICMP tracking disabled, no ICMP sessions allocated
IP tracking disabled, no IP sessions allocated
0 out of 1024 flowbits in use.

Packet Performance Monitor Config:
  ticks per usec  : 2417 ticks
  max packet time : 10000 usecs
  packet action   : fastpath-expensive-packets
  packet logging  : log
  debug-pkts      : disabled

Rule Performance Monitor Config:
  ticks per usec  : 2417 ticks
  max rule time   : 4096 usecs
  rule action     : suspend-expensive-rules
  rule threshold  : 5
  suspend timeout : 10 secs
  rule logging    : log
pcap DAQ configured to passive.
Acquiring network traffic from "em4".
Reload thread starting...
Reload thread started, thread 0xc100dbb8f00 (18056)
Decoding Ethernet

        --== Initialization Complete ==--

   ,,_     -*> Snort! <*-
  o"  )~   Version 2.9.4.6 GRE (Build 73)
   ''''    By Martin Roesch & The Snort Team:
http://www.snort.org/snort/snort-team
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using libpcap version 1.3.0
           Using PCRE version: 8.31 2012-07-06
           Using ZLIB version: 1.2.3

           Rules Engine: SF_SNORT_DETECTION_ENGINE  Version 1.17  <Build 18>
           Rules Object: web-misc  Version 1.0  <Build 1>
           Rules Object: web-iis  Version 1.0  <Build 1>
           Rules Object: web-client  Version 1.0  <Build 1>
           Rules Object: web-activex  Version 1.0  <Build 1>
           Rules Object: specific-threats  Version 1.0  <Build 1>
           Rules Object: snmp  Version 1.0  <Build 1>
           Rules Object: smtp  Version 1.0  <Build 1>
           Rules Object: p2p  Version 1.0  <Build 1>
           Rules Object: nntp  Version 1.0  <Build 1>
           Rules Object: netbios  Version 1.0  <Build 1>
           Rules Object: multimedia  Version 1.0  <Build 1>
           Rules Object: misc  Version 1.0  <Build 1>
           Rules Object: imap  Version 1.0  <Build 1>
           Rules Object: icmp  Version 1.0  <Build 1>
           Rules Object: exploit  Version 1.0  <Build 1>
           Rules Object: dos  Version 1.0  <Build 1>
           Rules Object: chat  Version 1.0  <Build 1>
           Rules Object: bad-traffic  Version 1.0  <Build 1>
           Preprocessor Object: SF_DNP3  Version 1.1  <Build 1>
           Preprocessor Object: SF_MODBUS  Version 1.1  <Build 1>
           Preprocessor Object: SF_GTP  Version 1.1  <Build 1>
           Preprocessor Object: SF_REPUTATION  Version 1.1  <Build 1>
           Preprocessor Object: SF_SIP  Version 1.1  <Build 1>
           Preprocessor Object: SF_SDF  Version 1.1  <Build 1>
           Preprocessor Object: SF_DCERPC2  Version 1.0  <Build 3>
           Preprocessor Object: SF_SSLPP  Version 1.1  <Build 4>
           Preprocessor Object: SF_DNS  Version 1.1  <Build 4>
           Preprocessor Object: SF_SSH  Version 1.1  <Build 3>
           Preprocessor Object: SF_SMTP  Version 1.1  <Build 9>
           Preprocessor Object: SF_IMAP  Version 1.0  <Build 1>
           Preprocessor Object: SF_POP  Version 1.0  <Build 1>
           Preprocessor Object: SF_FTPTELNET  Version 1.2  <Build 13>
Commencing packet processing (pid=18056)
^C*** Caught Int-Signal
===============================================================================
Run time for packet processing was 421.51287 seconds
Snort processed 630885 packets.
Snort ran for 0 days 0 hours 7 minutes 1 seconds
   Pkts/min:        90126
   Pkts/sec:         1498
===============================================================================
Packet Performance Summary:
   max packet time       : 10000 usecs
   packet events         : 0
   avg pkt time          : 5.9247 usecs
Rule Performance Summary:
   max rule time         : 4096 usecs
   rule events           : 0
===============================================================================
Packet I/O Totals:
   Received:      1863847
   Analyzed:       630885 ( 33.849%)
    Dropped:       601452 ( 24.397%)
   Filtered:            0 (  0.000%)
Outstanding:      1232962 ( 66.151%)
   Injected:            0
===============================================================================

Not really good numbers ....


Stats with only misc.rules and multimedia.rules (5 min, more or less):

Rule application order:
activation->dynamic->pass->drop->sdrop->reject->alert->log
Verifying Preprocessor Configurations!
ICMP tracking disabled, no ICMP sessions allocated
IP tracking disabled, no IP sessions allocated
WARNING: flowbits key 'file.vqf' is checked but not ever set.
WARNING: flowbits key 'file.wmp_playlist' is checked but not ever set.
8 out of 1024 flowbits in use.

[ Port Based Pattern Matching Memory ]
+- [ Aho-Corasick Summary ] -------------------------------------
| Storage Format    : Full-Q
| Finite Automaton  : DFA
| Alphabet Size     : 256 Chars
| Sizeof State      : Variable (1,2,4 bytes)
| Instances         : 27
|     1 byte states : 26
|     2 byte states : 1
|     4 byte states : 0
| Characters        : 1562
| States            : 1446
| Transitions       : 16926
| State Density     : 4.6%
| Patterns          : 90
| Match States      : 88
| Memory (KB)       : 562.24
|   Pattern         : 10.08
|   Match Lists     : 19.52
|   DFA
|     1 byte states : 261.06
|     2 byte states : 225.67
|     4 byte states : 0.00
+----------------------------------------------------------------
[ Number of patterns truncated to 20 bytes: 4 ]

Packet Performance Monitor Config:
  ticks per usec  : 2422 ticks
  max packet time : 10000 usecs
  packet action   : fastpath-expensive-packets
  packet logging  : log
  debug-pkts      : disabled

Rule Performance Monitor Config:
  ticks per usec  : 2422 ticks
  max rule time   : 4096 usecs
  rule action     : suspend-expensive-rules
  rule threshold  : 5
  suspend timeout : 10 secs
  rule logging    : log
pcap DAQ configured to passive.
Acquiring network traffic from "em4".
Reload thread starting...
Reload thread started, thread 0x4aa997dc00 (32237)
Decoding Ethernet

        --== Initialization Complete ==--

   ,,_     -*> Snort! <*-
  o"  )~   Version 2.9.4.6 GRE (Build 73)
   ''''    By Martin Roesch & The Snort Team:
http://www.snort.org/snort/snort-team
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using libpcap version 1.3.0
           Using PCRE version: 8.31 2012-07-06
           Using ZLIB version: 1.2.3

           Rules Engine: SF_SNORT_DETECTION_ENGINE  Version 1.17  <Build 18>
           Rules Object: web-misc  Version 1.0  <Build 1>
           Rules Object: web-iis  Version 1.0  <Build 1>
           Rules Object: web-client  Version 1.0  <Build 1>
           Rules Object: web-activex  Version 1.0  <Build 1>
           Rules Object: specific-threats  Version 1.0  <Build 1>
           Rules Object: snmp  Version 1.0  <Build 1>
           Rules Object: smtp  Version 1.0  <Build 1>
           Rules Object: p2p  Version 1.0  <Build 1>
           Rules Object: nntp  Version 1.0  <Build 1>
           Rules Object: netbios  Version 1.0  <Build 1>
           Rules Object: multimedia  Version 1.0  <Build 1>
           Rules Object: misc  Version 1.0  <Build 1>
           Rules Object: imap  Version 1.0  <Build 1>
           Rules Object: icmp  Version 1.0  <Build 1>
           Rules Object: exploit  Version 1.0  <Build 1>
           Rules Object: dos  Version 1.0  <Build 1>
           Rules Object: chat  Version 1.0  <Build 1>
           Rules Object: bad-traffic  Version 1.0  <Build 1>
           Preprocessor Object: SF_DNP3  Version 1.1  <Build 1>
           Preprocessor Object: SF_MODBUS  Version 1.1  <Build 1>
           Preprocessor Object: SF_GTP  Version 1.1  <Build 1>
           Preprocessor Object: SF_REPUTATION  Version 1.1  <Build 1>
           Preprocessor Object: SF_SIP  Version 1.1  <Build 1>
           Preprocessor Object: SF_SDF  Version 1.1  <Build 1>
           Preprocessor Object: SF_DCERPC2  Version 1.0  <Build 3>
           Preprocessor Object: SF_SSLPP  Version 1.1  <Build 4>
           Preprocessor Object: SF_DNS  Version 1.1  <Build 4>
           Preprocessor Object: SF_SSH  Version 1.1  <Build 3>
           Preprocessor Object: SF_SMTP  Version 1.1  <Build 9>
           Preprocessor Object: SF_IMAP  Version 1.0  <Build 1>
           Preprocessor Object: SF_POP  Version 1.0  <Build 1>
           Preprocessor Object: SF_FTPTELNET  Version 1.2  <Build 13>
Commencing packet processing (pid=32237)
^C*** Caught Int-Signal
===============================================================================
Run time for packet processing was 368.552024 seconds
Snort processed 643495 packets.
Snort ran for 0 days 0 hours 6 minutes 8 seconds
   Pkts/min:       107249
   Pkts/sec:         1748
===============================================================================
Packet Performance Summary:
   max packet time       : 10000 usecs
   packet events         : 0
   avg pkt time          : 8.95128 usecs
Rule Performance Summary:
   max rule time         : 4096 usecs
   rule events           : 0
   avg rule time         : 1.96408 usecs
===============================================================================
Packet I/O Totals:
   Received:      2121798
   Analyzed:       643495 ( 30.328%)
    Dropped:       618918 ( 22.582%)
   Filtered:            0 (  0.000%)
Outstanding:      1478303 ( 69.672%)
   Injected:            0
===============================================================================

About tunning sysctl options, if I am not wrong, OpenBSD tunes them
"on the fly" according to network load.

And more info: I have installed suricata in this host also to do more
tests, and suricata returns me best perfomance without losing many
packets:

-------------------------------------------------------------------
Counter                   | TM Name                   | Value
-------------------------------------------------------------------
capture.kernel_packets    | RxPcapem51                | 3052575199
capture.kernel_drops      | RxPcapem51                | 143259
capture.kernel_ifdrops    | RxPcapem51                | 0
decoder.pkts              | RxPcapem51                | 19561319
decoder.bytes             | RxPcapem51                | 15561225326
decoder.ipv4              | RxPcapem51                | 19561319
decoder.ipv6              | RxPcapem51                | 0
decoder.ethernet          | RxPcapem51                | 19561319
decoder.raw               | RxPcapem51                | 0
decoder.sll               | RxPcapem51                | 0
decoder.tcp               | RxPcapem51                | 19561139
decoder.udp               | RxPcapem51                | 0
decoder.sctp              | RxPcapem51                | 0
decoder.icmpv4            | RxPcapem51                | 180
decoder.icmpv6            | RxPcapem51                | 0
decoder.ppp               | RxPcapem51                | 0
decoder.pppoe             | RxPcapem51                | 0
decoder.gre               | RxPcapem51                | 0
decoder.vlan              | RxPcapem51                | 0
decoder.teredo            | RxPcapem51                | 0
decoder.ipv4_in_ipv6      | RxPcapem51                | 0
decoder.ipv6_in_ipv6      | RxPcapem51                | 0
decoder.avg_pkt_size      | RxPcapem51                | 796
decoder.max_pkt_size      | RxPcapem51                | 1506
defrag.ipv4.fragments     | RxPcapem51                | 0
defrag.ipv4.reassembled   | RxPcapem51                | 0
defrag.ipv4.timeouts      | RxPcapem51                | 0
defrag.ipv6.fragments     | RxPcapem51                | 0
defrag.ipv6.reassembled   | RxPcapem51                | 0
defrag.ipv6.timeouts      | RxPcapem51                | 0
defrag.max_frag_hits      | RxPcapem51                | 0
tcp.sessions              | Detect                    | 66702
tcp.ssn_memcap_drop       | Detect                    | 0
tcp.pseudo                | Detect                    | 7500
tcp.invalid_checksum      | Detect                    | 2
tcp.no_flow               | Detect                    | 0
tcp.reused_ssn            | Detect                    | 0
tcp.memuse                | Detect                    | 36175872
tcp.syn                   | Detect                    | 131466
tcp.synack                | Detect                    | 129929
tcp.rst                   | Detect                    | 56046
tcp.segment_memcap_drop   | Detect                    | 0
tcp.stream_depth_reached  | Detect                    | 306
tcp.reassembly_memuse     | Detect                    | 69060696
tcp.reassembly_gap        | Detect                    | 3214
detect.alert              | Detect                    | 38
flow_mgr.closed_pruned    | FlowManagerThread         | 78944
flow_mgr.new_pruned       | FlowManagerThread         | 3978
flow_mgr.est_pruned       | FlowManagerThread         | 2390
flow.memuse               | FlowManagerThread         | 3852512
flow.spare                | FlowManagerThread         | 10000
flow.emerg_mode_entered   | FlowManagerThread         | 0
flow.emerg_mode_over      | FlowManagerThread         | 0

Relevant data here are tcp.reassembly_gap and tcp.invalid_checksum numbers.

Any idea please??

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite
It's a free troubleshooting tool designed for production
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap2
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!
Previous By Date Next
Previous By Thread Next

Current thread: