Snort mailing list archives

Previous By Date Next
Previous By Thread Next

Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3

From: "C. L. Martinez" <carlopmart () gmail com>
Date: Thu, 6 Jun 2013 07:36:11 +0000
On Wed, Jun 5, 2013 at 5:08 PM, Victor Roemer <vroemer () sourcefire com> wrote:

Martinez, as Joel already mentioned, we'll want to see your Snort
configuration. Shutdown stats would also be useful, but perfmon data would
be better; if those can be provided.

You mentioned that OpenBSD configured the network sysctl parameters "on the
fly"; could you direct us to some documentation about this?

You also mentioned that Snort was listening on em3, however the startup
information in your email indicates that Snort is listening on em4, could
you elaborate on this setup?


Regarding Suricata, I personally do not have any experience in deploying or
configuring it. That said, are you using, relatively, the same
configurations? (e.g., any rules enabled, acquiring packets via libpcap,
etc..)

Also, why are "tcp.reassembly_gap" and "tcp.invalid_checksum" relevant?



Ok, first of all sorry for this long post. I will try to answer to all
questions. First, my environment:

Host: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz, 8 GiB RAM
Snort: release 2.9.4.6 compiled against libpcap 1.3.0
Suricata: release 1.4.2 compiled against libpcap 1.3.0

Snort is listening in em4 interface and suricata in em5 (there was a
typo error previously).

Performance data using my actual snort.conf (with few rules and
without so_rules or IP based rules):

Packet Performance Monitor Config:
  ticks per usec  : 2405 ticks
  max packet time : 10000 usecs
  packet action   : fastpath-expensive-packets
  packet logging  : log
  debug-pkts      : disabled

Rule Performance Monitor Config:
  ticks per usec  : 2405 ticks
  max rule time   : 4096 usecs
  rule action     : suspend-expensive-rules
  rule threshold  : 5
  suspend timeout : 10 secs
  rule logging    : log
pcap DAQ configured to passive.
Acquiring network traffic from "em4".
Reload thread starting...
Reload thread started, thread 0x1ee8c530a500 (4050)
Decoding Ethernet

        --== Initialization Complete ==--

   ,,_     -*> Snort! <*-
  o"  )~   Version 2.9.4.6 GRE (Build 73)
   ''''    By Martin Roesch & The Snort Team:
http://www.snort.org/snort/snort-team
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using libpcap version 1.3.0
           Using PCRE version: 8.31 2012-07-06
           Using ZLIB version: 1.2.3

           Rules Engine: SF_SNORT_DETECTION_ENGINE  Version 1.17  <Build 18>
           Preprocessor Object: SF_DNP3  Version 1.1  <Build 1>
           Preprocessor Object: SF_MODBUS  Version 1.1  <Build 1>
           Preprocessor Object: SF_GTP  Version 1.1  <Build 1>
           Preprocessor Object: SF_REPUTATION  Version 1.1  <Build 1>
           Preprocessor Object: SF_SIP  Version 1.1  <Build 1>
           Preprocessor Object: SF_SDF  Version 1.1  <Build 1>
           Preprocessor Object: SF_DCERPC2  Version 1.0  <Build 3>
           Preprocessor Object: SF_SSLPP  Version 1.1  <Build 4>
           Preprocessor Object: SF_DNS  Version 1.1  <Build 4>
           Preprocessor Object: SF_SSH  Version 1.1  <Build 3>
           Preprocessor Object: SF_SMTP  Version 1.1  <Build 9>
           Preprocessor Object: SF_IMAP  Version 1.0  <Build 1>
           Preprocessor Object: SF_POP  Version 1.0  <Build 1>
           Preprocessor Object: SF_FTPTELNET  Version 1.2  <Build 13>
Commencing packet processing (pid=4050)
===============================================================================
Run time for packet processing was 368.552024 seconds
Snort processed 643495 packets.
Snort ran for 0 days 0 hours 6 minutes 8 seconds
   Pkts/min:       107249
   Pkts/sec:         1748
===============================================================================
Packet Performance Summary:
   max packet time       : 10000 usecs
   packet events         : 0
   avg pkt time          : 8.95128 usecs
Rule Performance Summary:
   max rule time         : 4096 usecs
   rule events           : 0
   avg rule time         : 1.96408 usecs
===============================================================================
Packet I/O Totals:
   Received:      2121798
   Analyzed:       643495 ( 30.328%)
    Dropped:       618918 ( 22.582%)
   Filtered:            0 (  0.000%)
Outstanding:      1478303 ( 69.672%)
   Injected:            0
===============================================================================

And here they are some statistics outputs:

top output:

load averages:  0.18,  0.18,  0.13
21 processes: 20 idle, 1 on processor
CPU0 states:  0.0% user,  0.0% nice,  0.0% system,  0.6% interrupt, 99.4% idle
CPU1 states:  1.0% user,  0.0% nice,  0.0% system,  0.0% interrupt, 99.0% idle
CPU2 states:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU3 states:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
Memory: Real: 506M/2942M act/tot Free: 5018M Cache: 2319M Swap: 0K/6142M

  PID USERNAME PRI NICE  SIZE   RES STATE     WAIT      TIME    CPU COMMAND
 4050 root       4    0  417M  151M sleep/0   bpf       0:06  2.83% snort
31522 root      10    0  356M  328M sleep/0   nanosle  35:09  0.73% suricata

Status interfaces (systat ifstat):

    2 users    Load 0.17 0.17 0.13

IFACE            STATE  DESC
                     IPKTS   IBYTES    IERRS    OPKTS   OBYTES
OERRS    COLLS
em0              up
                         1       72        0        0       71
0        0
em1              up
                         0        0        0        0        0
0        0
em2              up
                         0        0        0        0        0
0        0
em3              up
                         1      334        0        2     1732
0        0
em4              up
                      6773  4771755        0        0        0
0        0
em5              up
                      6773  4771755        0        0        0
0        0

Interfaces configruation options:

em4: flags=8bc3<UP,BROADCAST,RUNNING,NOARP,PROMISC,ALLMULTI,SIMPLEX,MULTICAST>
mtu 1514
        lladdr 00:50:56:17:fc:cd
        priority: 0
        media: Ethernet autoselect (1000baseT full-duplex,master)
        status: active
        inet6 fe80::250:56ff:fe17:fccd%em4 prefixlen 64 scopeid 0x5
em5: flags=8bc3<UP,BROADCAST,RUNNING,NOARP,PROMISC,ALLMULTI,SIMPLEX,MULTICAST>
mtu 1514
        lladdr 00:50:56:18:79:51
        priority: 0
        media: Ethernet autoselect (1000baseT full-duplex,master)
        status: active
        inet6 fe80::250:56ff:fe18:7951%em5 prefixlen 64 scopeid 0x6

Checking mbuffers (systat mbufs):

    2 users    Load 0.18 0.17 0.13

IFACE             LIVELOCKS  SIZE ALIVE   LWM   HWM   CWM
System                    0   256   361          59
                               2k   351         457
lo0
em0                            2k     7     4   256     7
em1                            2k     6     4   256     6
em2                            2k     4     4   256     4
em3                            2k     7     4   256     7
em4                            2k    69     4   256    69
em5                            2k    73     4   256    73

sysctl.conf options enabled:
kern.bufcachepercent=80
net.bpf.bufsize=65536
net.bpf.maxbufsize=2097152

Ok, now some Suricata stats:

5/6/2013 -- 14:29:09 - <Info> - stream "max-sessions": 262144
5/6/2013 -- 14:29:09 - <Info> - stream "prealloc-sessions": 32768
5/6/2013 -- 14:29:09 - <Info> - stream "memcap": 33554432
5/6/2013 -- 14:29:09 - <Info> - stream "midstream" session pickups: disabled
5/6/2013 -- 14:29:09 - <Info> - stream "async-oneside": disabled
5/6/2013 -- 14:29:09 - <Info> - stream "checksum-validation": enabled
5/6/2013 -- 14:29:09 - <Info> - stream."inline": disabled
5/6/2013 -- 14:29:09 - <Info> - stream.reassembly "memcap": 67108864
5/6/2013 -- 14:29:09 - <Info> - stream.reassembly "depth": 1048576
5/6/2013 -- 14:29:09 - <Info> - stream.reassembly "toserver-chunk-size": 2560
5/6/2013 -- 14:29:09 - <Info> - stream.reassembly "toclient-chunk-size": 2560
5/6/2013 -- 14:29:09 - <Info> - all 7 packet processing threads, 3
management threads initialized, engine started.
5/6/2013 -- 14:29:13 - <Info> - No packets with invalid checksum,
assuming checksum offloading is NOT used
6/6/2013 -- 06:27:20 - <Info> - Signal Received.  Stopping engine.
6/6/2013 -- 06:27:20 - <Info> - 0 new flows, 0 established flows were
timed out, 0 flows in closed state
6/6/2013 -- 06:27:21 - <Info> - time elapsed 57492.090s
6/6/2013 -- 06:27:21 - <Info> - (RxPcapem51) Packets 7317731, bytes 5500227933
6/6/2013 -- 06:27:21 - <Info> - (RxPcapem51) Pcap Total:2974089715
Recv:2974089715 Drop:0 (0.0%).
6/6/2013 -- 06:27:21 - <Info> - AutoFP - Total flow handler queues - 6
6/6/2013 -- 06:27:21 - <Info> - AutoFP - Queue 0  - pkts: 2754800
flows: 19921
6/6/2013 -- 06:27:21 - <Info> - AutoFP - Queue 1  - pkts: 2158501
flows: 15267
6/6/2013 -- 06:27:21 - <Info> - AutoFP - Queue 2  - pkts: 1276685
flows: 9932
6/6/2013 -- 06:27:21 - <Info> - AutoFP - Queue 3  - pkts: 678835
flows: 5843
6/6/2013 -- 06:27:21 - <Info> - AutoFP - Queue 4  - pkts: 358226
flows: 3109
6/6/2013 -- 06:27:21 - <Info> - AutoFP - Queue 5  - pkts: 127487
flows: 1488

-------------------------------------------------------------------
Date: 6/6/2013 -- 06:27:21 (uptime: 0d, 15h 58m 44s)
-------------------------------------------------------------------
Counter                   | TM Name                   | Value
-------------------------------------------------------------------
capture.kernel_packets    | RxPcapem51                | 2974088271
capture.kernel_drops      | RxPcapem51                | 0
capture.kernel_ifdrops    | RxPcapem51                | 0
decoder.pkts              | RxPcapem51                | 7317731
decoder.bytes             | RxPcapem51                | 5500227933
decoder.ipv4              | RxPcapem51                | 7317731
decoder.ipv6              | RxPcapem51                | 0
decoder.ethernet          | RxPcapem51                | 7317731
decoder.raw               | RxPcapem51                | 0
decoder.sll               | RxPcapem51                | 0
decoder.tcp               | RxPcapem51                | 7317660
decoder.udp               | RxPcapem51                | 0
decoder.sctp              | RxPcapem51                | 0
decoder.icmpv4            | RxPcapem51                | 71
decoder.icmpv6            | RxPcapem51                | 0
decoder.ppp               | RxPcapem51                | 0
decoder.pppoe             | RxPcapem51                | 0
decoder.gre               | RxPcapem51                | 0
decoder.vlan              | RxPcapem51                | 0
decoder.teredo            | RxPcapem51                | 0
decoder.ipv4_in_ipv6      | RxPcapem51                | 0
decoder.ipv6_in_ipv6      | RxPcapem51                | 0
decoder.avg_pkt_size      | RxPcapem51                | 752
decoder.max_pkt_size      | RxPcapem51                | 1506
defrag.ipv4.fragments     | RxPcapem51                | 0
defrag.ipv4.reassembled   | RxPcapem51                | 0
defrag.ipv4.timeouts      | RxPcapem51                | 0
defrag.ipv6.fragments     | RxPcapem51                | 0
defrag.ipv6.reassembled   | RxPcapem51                | 0
defrag.ipv6.timeouts      | RxPcapem51                | 0
defrag.max_frag_hits      | RxPcapem51                | 0
tcp.sessions              | Detect                    | 41460
tcp.ssn_memcap_drop       | Detect                    | 0
tcp.pseudo                | Detect                    | 5705
tcp.invalid_checksum      | Detect                    | 12
tcp.no_flow               | Detect                    | 0
tcp.reused_ssn            | Detect                    | 0
tcp.memuse                | Detect                    | 36175872
tcp.syn                   | Detect                    | 80827
tcp.synack                | Detect                    | 80035
tcp.rst                   | Detect                    | 33845
tcp.segment_memcap_drop   | Detect                    | 0
tcp.stream_depth_reached  | Detect                    | 148
tcp.reassembly_memuse     | Detect                    | 67770240
tcp.reassembly_gap        | Detect                    | 2222
detect.alert              | Detect                    | 29
flow_mgr.closed_pruned    | FlowManagerThread         | 49711
flow_mgr.new_pruned       | FlowManagerThread         | 3558
flow_mgr.est_pruned       | FlowManagerThread         | 1974
flow.memuse               | FlowManagerThread         | 3884096
flow.spare                | FlowManagerThread         | 10001
flow.emerg_mode_entered   | FlowManagerThread         | 0
flow.emerg_mode_over      | FlowManagerThread         | 0

As you can see, packets dropped counter shows 0% (well, in fact this
is not always the case. Suricata may lost between 1-2% of packets).
And this suricata sensor is configured to use IP based rules, when
snort not.
But there are two counters that can shows if some problems exists:
tcp.reassembly_gap and tcp.invalid_checksum. But as you can see, are
minimal according to Suricata docs:

https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Statistics

When I say: "OpenBSD tunes them "on the fly" according to network
load", I would say that OpenBSD 5.1 and later will dynamically adjust
the TCP send and receive window sizes. Please, refer to:

http://marc.info/?l=openbsd-misc&m=130804020930481&w=2
http://marc.info/?l=openbsd-misc&m=128904951710762&w=2
http://www.openbsd.org/faq/faq6.html#Tuning
http://quigon.bsws.de/papers/2013/AsiaBSDcon/cksums.pdf

Do you need to add more info or perform more tests?

And many thanks for your help.

Attachment: snort.conf
Description: 

------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. A cloud service to automate IT design, transition and operations
2. Dashboards that offer high-level views of enterprise services
3. A single system of record for all IT processes
http://p.sf.net/sfu/servicenow-d2d-j
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!
Previous By Date Next
Previous By Thread Next

Current thread: