Re: Performance impact with multiple pcap handlers on Linux

[Guy Harris via tcpdump-workers <tcpdump-workers () lists tcpdump org>]

> --- Begin Message ---
>
>
> From: Guy Harris <gharris () sonic net>
>
> Date: Tue, 22 Dec 2020 14:28:17 -0800
>
> On Dec 22, 2020, at 2:05 PM, Linus Lüssing via tcpdump-workers <tcpdump-workers () lists tcpdump org> wrote:
>
> > I was experimenting a bit with migrating from the use of
> > pcap_offline_filter() to pcap_setfilter().
> >
> > I was a bit surprised that installing for instance 500 pcap
> > handlers
>
> What is a "pcap handler" in this context?  An open live-capture pcap_t?
>
> > with a BPF rule "arp" via pcap_setfilter() reduced
> > the TCP performance of iperf3 over veth interfaces from 73.8 Gbits/sec
> > to 5.39 Gbits/sec. Using only one or even five handlers seemed
> > fine (71.7 Gbits/sec and 70.3 Gbits/sec).
> >
> > Is that expected?
> >
> > Full test setup description and more detailed results can be found
> > here: https://github.com/lemoer/bpfcountd/pull/8
>
> That talks about numbers of "rules" rather than "handlers".  It does speak of "pcap *handles*"; did you mean "handles", 
> rather than "handlers"?
>
> Do those "rules" correspond to items in the filter expression that's compiled into BPF code, or do they correspond to 
> open `pcap_t`s?  If a "rule" corresponds to a "handle", then does it correspond to an open pcap_t?
>
> Or do they correspond to an entire filter expression?
>
> Does this change involve replacing a *single* pcap_t, on which you use pcap_offline_filter() with multiple different 
> filter expressions, with *multiple* pcap_t's, with each one having a separate filter, set with pcap_setfilter()?  If 
> so, note that this involves replacing a single file descriptor with multiple file descriptors, and replacing a single 
> ring buffer into which the kernel puts captured packets with multiple ring buffers into *each* of which the kernel puts 
> captured packets, which increases the amount of work the kernel does.
>
> > PS: And I was also surprised that there seems to be a limit of
> > only 510 pcap handlers on Linux.
>
> "handlers" or "handles"?
>
> If it's "handles", as in "pcap_t's open for live capture", and if you're switching from a single pcap_t to multiple 
> pcap_t's, that means using more file descriptors (so that you may eventually run out) and more ring buffers (so that 
> the kernel may eventually say "you're tying up too much wired memory for all those ring buffers").
>
> In either of those cases, the attempt to open a pcap_t will eventually get an error; what is the error that's reported?
>
> --- End Message ---
_______________________________________________
tcpdump-workers mailing list
tcpdump-workers () lists tcpdump org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers