Re: Service Provider NetFlow Collectors

[Mark Tinka <mark.tinka () seacom mu>]

We were on Arbor for quite some time, but are now moving to Kentik.

Mark.

On 3/Jan/19 05:37, Nick Peelman wrote:
> We rolled a large(ish) ElasticSearch cluster last year out of SuperMicro Microclouds (3U, 8 nodes per chassis, Xeon-D 
> based processors), mostly 32GB of RAM per node, and M.2 PCIe SSDs as well as HDD storage.  ES is a finicky beast to 
> maintain. It can handle a node completely dying or disappearing from the network, but not when one runs out of space 
> (at least not gracefully).  Maintaining retention and rotation is tedious at best (yay curator).  We’re dumping a 
> boatload of log data there, as well as Flow data using Elastiflow, which provides the necessary collector bits as 
> well as all the pretty Kibana graphs and stuff.  Probably overbuilt, but I can pretty much keep whatever logs we want 
> in perpetuity, we have plenty of headroom, and searching is incredibly fast.
>
> ELK is an awesome set of tools, but be warned, there be dragons.  Admin’ing even a small cluster can be time 
> consuming and frustrating, and requires a pretty stout linux and server background, or at least some really good 
> troubleshooting skills and an ability to turn to the code when the docs fall short.  Doing a larger cluster could 
> easily be a full time job.  Still, all in all, I’m happy with the cost of ours, including my time building it and 
> continued time maintaining it, compared to what the yearly outlay was going to be for Kentik.
>
> -nick
>
> On 31 Dec 2018, at 11:40, Mike Hammett <nanog () ics-il net<mailto:nanog () ics-il net>> wrote:
>
> I just recently rolled out Elastiflow. Lots of great information.
>
>
>
> -----
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com<http://www.ics-il.com/>
>
> Midwest-IX
> http://www.midwest-ix.com<http://www.midwest-ix.com/>
>
> ________________________________
> From: "Michel 'ic' Luczak" <lists () benappy com<mailto:lists () benappy com>>
> To: "Erik Sundberg" <ESundberg () nitelusa com<mailto:ESundberg () nitelusa com>>
> Cc: nanog () nanog org<mailto:nanog () nanog org>
> Sent: Monday, December 31, 2018 3:40:40 AM
> Subject: Re: Service Provider NetFlow Collectors
>
> Don’t underestimate good old ELK
> https://www.elastic.co/guide/en/logstash/current/netflow-module.html
> + https://github.com/robcowart/elastiflow
>
> BR, ic
>
> On 31 Dec 2018, at 04:29, Erik Sundberg <ESundberg () nitelusa com<mailto:ESundberg () nitelusa com>> wrote:
>
> Hi Nanog….
>
> We are looking at replacing our Netflow collector. I am wonder what other service providers are using to collect 
> netflow data off their Core and Edge Routers. Pros/Cons… What to watch out for any info would help.
>
> We are mainly looking to analyze the netflow data. Bonus if it does ddos detection and mitigation.
>
> We are looking at
> ManageEngine Netflow Analyzer
> PRTG
> Plixer – Scrutinizer
> PeakFlow
> Kentik
> Solarwinds NTA
>
>
> Thanks in advance…
>
> Erik
>
>
> ________________________________
>
> CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it 
> may contain confidential information that is legally privileged. If you are not the intended recipient, or a person 
> responsible for delivering it to the intended recipient, you are hereby notified that any disclosure, copying, 
> distribution or use of any of the information contained in or attached to this transmission is STRICTLY PROHIBITED. 
> If you have received this transmission in error please notify the sender immediately by replying to this e-mail. You 
> must destroy the original transmission and its attachments without reading or saving in any manner. Thank you.