Re: high packet loss - low throughput

[waldo kitty <wkitty42 () windstream net>]

On 7/19/2013 15:51, Michal Purzynski wrote:
> On 7/19/13 6:37 PM, waldo kitty wrote:
> > On 7/19/2013 05:16, Michal Purzynski wrote:
> > > So, anyone got some ideas how to debug and improve the situation? Or
> > > should I just assume that snort isn't capable of handling a per process
> > > 30Mbit - I can see a 5% packet loss now.
> > are you running a 64bit OS on those boxes or a 32bit one? which OS? you said
> > (below) part of a security onion so i'm going to guess linux... now 64 or 32bit?
> >
> > assuming *nix, what does top show?
> >
> >      top -bn1 | head
> top -bn1 | head
>
> top - 19:50:58 up 10:15,  1 user,  load average: 5.74, 5.30, 4.59

you definitely have something keeping the processors busy... let's run that top 
command again with a larger head...

top -bn1 | head 20

> Tasks: 321 total,   3 running, 318 sleeping,   0 stopped,   0 zombie
> Cpu(s): 15.5%us,  0.8%sy,  0.0%ni, 81.7%id,  0.1%wa,  0.0%hi, 1.9%si,
> 0.0%st

well, it isn't disk bound (wa) which is good... i wouldn't expect it to be 
either... all of these seems to be ok i think... just not sure about 15.5% being 
spent in user time but i guess that depends on the apps running and as what 
user... looking at the below, i see that snort is running as user sguil so that 
may be part of the deal on that one...

> Mem:  65939336k total, 65673944k used,   265392k free,    33508k buffers
> Swap: 33969596k total,        0k used, 33969596k free, 46105348k cached

no swap used which is good...

>     PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+ COMMAND
> 33155 sguil     20   0  379m 354m 337m R   39  0.5 238:13.59 netsniff-ng
> 34035 sguil     20   0  849m 727m  11m R   33  1.1 102:38.41 snort
> 35091 sguil     20   0  851m 731m  11m S   25  1.1 111:49.86 snort

we'll see more of these with the above command... i note that netsniff-ng has 
twice as much time consumed as the shown snort processes... what happens if you 
disable netsniff-ng?

> 64 bit of course. It's Ubuntu 12.04.2, everything updated, etc.

ok...

> I've noticed an interesting statistics BTW:
> - there are some processes doing +- 60Mbit/sec with a packet loss over 6%
> - there are some doing 90-100 with 0% packet loss (or at least below 1%,
> which is my goal)
>
> I don't understand it, what might be a reason?

not sure yet but something is binding things down... here's a bassackwards 
thought... i asked if you had tried running more instances to see if each 
additional instance would take some of the load off the others and allow all of 
them to process more... you said you had gone from 6 to 8... my first thought 
was to try more... if you were running 6 then double it and see what happens... 
now i'm wondering what happens if you drop the number of instances back so that 
each processes more but there are fewer to keep the system busy... what happens 
if you run only one instance? then two instances? and so on...

also, what are your NICs? do they have hardware offloading capabilities for some 
functions so that snort and/or your OS doesn't have to deal with them? are you 
using those capabilities?

-- 
NOTE: No off-list assistance is given without prior approval.
       Please keep mailing list traffic on the list unless
       private contact is specifically requested and granted.

------------------------------------------------------------------------------
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!