Re: RE : Re: high packet loss - low throughput

[Michal Purzynski <michal () rsbac org>]

On 7/19/13 2:32 PM, rmkml wrote:
> Hi Michal,
>
> Sorry if I don't followed your all answers,
>
> What's cpu if you run all snort with "special" bpf for testing 
> interrupt/network driver/pfring please? (Bpf like "tcp port 79")
> Send top result ?
It's in this email already.
> Can you run a snort output statistics after one minute please?  After 
> 5mn ?
Sure, snort takes no more than 20-40% CPU, with a short spikes.
> It's a new snort install or It's a snort upgrade? What cpu previously?
New install.
> What's os you use please?  Tunning? Sysctl ?
Nothing.
> What's cpu if you run all snort without bpf and without rules/module 
> please?
>
> Can you replace snort by tcpdump only for testing?  Cpu results?
Errr?
> Regards
> @Rmkml
>
>
>
>
>
> -------- Message d'origine --------
> De : Michal Purzynski <michal () rsbac org>
> Date :
> A : snort-users () lists sourceforge net
> Objet : Re: [Snort-users] high packet loss - low throughput
>
>
> So, anyone got some ideas how to debug and improve the situation? Or
> should I just assume that snort isn't capable of handling a per process
> 30Mbit - I can see a 5% packet loss now.
>
> On 7/18/13 11:07 AM, Michal Purzynski wrote:
> > On 7/18/13 3:39 AM, waldo kitty wrote:
> >> On 7/17/2013 17:25, Michal Purzynski wrote:
> >>> On 7/17/13 11:01 PM, waldo kitty wrote:
> >>>> On 7/17/2013 16:04, Michal Purzynski wrote:
> >>>>> Hello,
> >>>>>
> >>>>> I can see a strange results on a local snort installation. Either I
> >>>>> don't understand something or the statistics aren't precise.
> >>>>> Please help
> >>>>> me understand.
> >>>>>
> >>>>> It's an (expanding) two hosts snort setup with 2 x E5-2620 0 @
> >>>>> 2.00GHz /
> >>>>> 64GB RAM each.
> >>>>> Intel x520 card.
> >>>>> Traffic is around 1Gbit to each host.
> >>>>> Around 3500 VRT only rules enabled.
> >>>>> 8 snort instances load balanced by the pf_ring.
> >>>> what else is this machine doing besides just snorting the traffic?
> >>> netsniff-ng, barnyard, snort and that's it. Part of a Security Onion,
> >>> but with most things (like Bro, argus, prads, etc) disabled.
> >>>>> The traffic loss is very high - up to 9% per instance (as 
> reported by
> >>>>> Sguil which in turn read the snort logs and debug files). A single
> >>>>> instance gets from 90 - 150Mbits of traffic and from 10 - 20k 
> pps. To
> >>>>> make it worse, the loss is not dependent on the traffic and/or 
> pps at
> >>>>> all. Actualy, sometimes I get a 5% of loss on 50Mbits to a single
> >>>>> instance.
> >>>> what happens if you increase the number of snort instances which
> >>>> would thereby
> >>>> reduce the load on each of the instances?
> >>> I did it increasing from 6 to 8. And it won't help, really - if snort
> >>> cannot keep up with 50Mbit / instance stream...
> >> i'm not sure that it is snort, specifically... there is something
> >> causing the
> >> data to be flushed or lost before it has a chance to be processed...
> >> there are
> >> others running snort on pipes as large or larger...
> >>
> >> perhaps you are using protocol aware stream flushing and it needs
> >> tweaking?
> > Yes, it's enabled with the same settings. Reading about it and I don't
> > really want to disable it.
> >>
> >> ###############################################
> >> # Configure protocol aware flushing
> >> # For more information see README.stream5
> >> ###############################################
> >> config paf_max: 16000
> >>
> >>
> >> it may also be related to the timeout values in the stream5 settings??
> >>
> >>
> > No idea, that's why asking here :) Everything is default.
>
>
> ------------------------------------------------------------------------------
> See everything from the browser to the database with AppDynamics
> Get end-to-end visibility with application monitoring from AppDynamics
> Isolate bottlenecks and diagnose root cause in seconds.
> Start your free trial of AppDynamics Pro today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
>
> Please visit http://blog.snort.org to stay current on all the latest 
> Snort news!

------------------------------------------------------------------------------
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk

_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!