Re: End to End testing

[Saku Ytti <saku () ytti fi>]

On Thu, 12 Dec 2019 at 16:56, Fawcett, Nick via NANOG <nanog () nanog org> wrote:

> Anyone have any suggestions on devices that I can put at two points in the network to test packet loss, latency, 
> jitter etc.  I was thinking of maybe engineering my own using a couple of pi’s,  but the downfall is they don’t have 
> SFP ports.  I’m looking for something that’s portable and easy to configure and drop in.  Thanks.

You may need to consider what are your requirements

a) how many PPS you want (that is, what is smallest outage you want to
be sure you know about?).
b) how often do you want to generate database row out of the probed packets
c) what probe-to-render delay you need, so when something happens,
when should it be visible in UX? (rapid time is useful for
troubleshooting, if you get feedback in seconds, many solutions have
very long time before results are visible)
d) what type of precision you need, does it need to be same for 1way and 2way
e) do you want to use only vendor stack, or do you also want to stream
the data to your own bigdata
f) do you want to use active probes and generate the packets you use
to measure or do you want have in-path devices to measure real
production traffic
g) do you need some solution to ECMP or are you content that your
measurement only sees one LAG/ECMP option across entire network
(almost no one has any solution here, and most networks these days
tend to balance every single link)


There are quite few solutions there, for me only active probes were
considered. Finance world seems to like in-path devices looking real
packets, but there you don't know intent, you don't know where the
packet was supposed to go, active probe knows that too.
I looked at every product I could find (googling, j-nsp, c-nsp, nanog,
finding vendors googling different combinations of vendors to find
more vendors). And ended up shortlisting Accedian, Netrounds and
Creanord, and out of those I'm not sure netrounds would have worked
for me, I'm pretty sure Accedian would have, but ended up going with
Creanord (CreaNode probes and Echovault UX/DB) with also Kafka
streaming to our own DB. We measure three QoS classes full mesh with
over 100 pops sending from every QoS class to every other pop every
10ms and uploading row of data every 10s. 2way precision is better
than 1us (so about 200m fiber length precision in measurements), 1way
is NTP limited so some hundreds of us.

Any solution you choose be sure you have plans on how to capitalise
and use the data. It is relatively easy to produce a lot of data, but
if you don't use it, it's a lot of money and effort wasted.

-- 
  ++ytti