Re: Open source Netflow analysis for monitoring AS-to-AS traffic

[Javier Gutierrez <GutierrezJ () westmancom com>]

After some troubleshooting I ended up having to increase my kafka partitions as well as my clickhouse collectors as it 
seemed like clickhouse would lack behind quite a bit
I also has some issues with the server where CPU and RAM would max out all the time, my RAM usage is still quite high 
and seems to grow exponentially as the day goes by, but i don't think its causing the issues anymore. Storage wise I'm 
good tho

Thanks for the advice.


Kind regards,



Javier Gutierrez,

________________________________
From: Vincent Bernat <bernat () luffy cx>
Sent: Saturday, June 8, 2024 2:46 AM
To: Javier Gutierrez <GutierrezJ () westmancom com>; Steven Bakker <steven.bakker () ams-ix net>; Peter Phaal 
<peter.phaal () gmail com>
Cc: nanog () nanog org <nanog () nanog org>
Subject: Re: Open source Netflow analysis for monitoring AS-to-AS traffic

CAUTION: This email is from an external source. Do not click links or open attachments unless you recognize the sender 
and know the content is safe.

Without much information, I think this is more likely that you are
running out of disk space.

On 2024-06-05 23:15, Javier Gutierrez wrote:
> Hi everyone,
> I've been trying to get Akvorado to work on my environmnet but I keep
> getting the flows to stop collecting, it seems like the issue is related
> to the number of exporters I have sending data, can someone please share
> the max number they have gotten to work and the flows/s rate without the
> system crashing?
>
> Thanks in advance for your answers.
> ------------------------------------------------------------------------
> *From:* NANOG <nanog-bounces+gutierrezj=westmancom.com () nanog org> on
> behalf of Steven Bakker <steven.bakker () ams-ix net>
> *Sent:* Sunday, March 31, 2024 4:53 AM
> *To:* Peter Phaal <peter.phaal () gmail com>
> *Cc:* nanog () nanog org <nanog () nanog org>
> *Subject:* Re: Open source Netflow analysis for monitoring AS-to-AS traffic
>
> *CAUTION: *This email is from an external source. Do not click links or
> open attachments unless you recognize the sender and know the content is
> safe.
>
> Hi Peter,
>
> Thanks for that link. I did read the spec, and while the definition
> itself is clear, the escape clause gives a lot of wiggle room:
>
> "/Hardware limitations may// prevent an exact reporting of the
> underlying frame length, but an agent should attempt to //be as accurate
> as possible./"
>
> I read that as, "the vendor will do whatever it pleases, and you should
> be grateful to receive a non-negative integer at all." I could be too
> cynical, though.
>
> Anyway, this particular vendor does other funny things (such as
> sometimes stripping the q-tag headers from the sampled frame; throttling
> the frame sampling on the box, but not adjusting the sampling interval
> in the sFlow exports) that make it a true joy to work with this gear. ;-)
>
> Cheers,
>
> -- Steven