Re: RFC: a loadable plugin framework for external capture backends

[Robin Jarry via tcpdump-workers <tcpdump-workers () lists tcpdump org>]

> --- Begin Message ---
>
>
> From: "Robin Jarry" <rjarry () redhat com>
>
> Date: Thu, 01 Oct 2026 23:53:53 +0200
>
> Hi Michael,
>
> Important disclaimer first; I should have stated this in my original 
> post: DPDK is not an *application* nor a *networking stack*. It is an 
> SDK exposing libraries and native hardware drivers to *build* 
> applications.
>
> On the other hand, grout, OVS (and VPP to some extent) are applications 
> built around DPDK. The only thing these applications have in common is 
> that they bypass the kernel stack. They don't share any API. This is the 
> reason why it is close to impossible to share a traffic capture API with 
> all userspace stacks. They all have a different representation of what 
> is an interface, what is a packet, etc.
>
> Michael Richardson, Oct 01, 2026 at 22:33:
> > So, I want to distinguish between a **plugin** vs a stable API/ABI 
> > that DPDK, VPP, etc. could work against.  Or a stable ABI that libpcap 
> > could work against.
>
> I want to turn this around. libpcap defines the API/ABI and does not 
> care about who consumes it. It just requires that third-party plugins 
> implement it. This means close to zero maintenance cost on libpcap and 
> maximum flexibility for external backends.
>
> > > Modern userspace dataplanes (DPDK-based routers such as grout, 
> > > OVS-DPDK, VPP, and similar) move packets entirely in userspace. Their 
> > > interfaces are invisible to the kernel, so `tcpdump -i <iface>` 
> > > cannot open them as a live source. Capturing traffic from a running 
> > > dataplane today means either a bespoke CLI tool per project, or 
> > > building capture support directly into libpcap.
> >
> > If we can create an interface which is not tied to a specific (DPDK) 
> > version, then I prefer that the support be statically built in.
>
> I don't get what you mean here. Statically built in libpcap? This is 
> what I'd like to avoid specifically. Not tie the fast-moving backend to 
> a stable libpcap.
>
> > The rpcapd solution should not be horrible if the percent of traffic 
> > is low, and the BPF program runs inside of the user-space engine. This 
> > might be the best story for "casual" captures. It won't due for 
> > someone trying to eat 100Gb/s of live DNS traffic for analysis. 
> > https://adam.pages.nic.cz/dns-probe/Running.html does not, AFAIK, use 
> > libpcap for these preformance reasons, offering TPACKET or DPDK 
> > integrations.
> >
> > I think that the rpcap protocol *itself* could be better, mind you.
>
> I don't think the rpcap protocol is the issue here. The issue is 
> allowing system calls in a high performance datapath. Even if it is 
> transient and for debugging purposes, the side effects and resulting 
> latency can be disastrous.
>
> > > This is the same reason PAM, NSS, GStreamer, OpenSSL providers and 
> > > Mesa all converged on a loadable-module model.
> >
> > Some of these are good examples, and there have been problems with all 
> > of them over the decades :-)
> >
> > NSS and PAM have had many vulnerabilities/gotchas in decades past. (I 
> > also have many unpritable opinions about the OpenSSL provider 
> > interfaces)
>
> Indeed nothing is perfect. But my hope is that we can find enough good 
> examples that have proven to work and where most vulnerabilities have 
> been fixed over the years.
>
> > So, let me suggest different interfaces which are relatively stable, 
> > and are already shared memory based: TPACKET_V3 and AF_XDP:
> >   https://www.kernel.org/doc/html/latest/networking/af_xdp.html
> >
> > I recognize that these are two different but very similiar things, and 
> > I am unclear to what extent the Linux kernel treats them in similiar 
> > code paths. I believe that VPP, DPDK, etc. use AF_XDP to get packets 
> > from the kernel, and send them back to the device.  (I think that 
> > there are also still vendor proprietary paths?)
> >
> > There is a win here in that libpcap could wind up with an AF_XDP 
> > capture path.  Obviously, the setsockopt() calls that AF_XDP requires 
> > have to replaced with a different communication.  Probably a UNIX 
> > domain socket, likely the shared memory segments used for the packets 
> > and rings would be passed by fd through that socket.   (Which way? 
> > Probably both)
> >
> > I think, but I can not be certain, that this might permit zero-copy 
> > loans of the packet buffers from VPP/DPDK/etc. to libpcap. Read-only? 
> > Copy-on-Write maybe useful of the engine needs to mutate the data 
> > while libpcap is still processing... but likely that's too expensive.
>
> If you care about performance, you will likely *not* use AF_XDP. It 
> looks seducing at first glance: you don't have to reimplement any 
> hardware specific interaction in userspace, just create a socket and 
> mmap() it to receive packets.
>
> In practice, you can get the same level of performance with AF_XDP 
> compared to native DPDK userspace drivers... with twice as many CPUs.
>
> I am exaggerating on purpose here, but the TLDR is that anything 
> touching the kernel will not be satisfactory if you expect to forward 
> 15+ million packets per second on a single CPU.
>
> > > - Enumeration vs selection. Do plugins only contribute devices to 
> > >   pcap_findalldevs(), or can they intercept arbitrary device names?
> > >
> > >   Our preference: a plugin owns a namespace prefix (e.g. "grout:") 
> > >   and is consulted only for names in its namespace. This makes "how 
> > >   do multiple plugins coexist" answerable by construction.
> >
> > So, the devices would not show up for --list-interfaces, but if one 
> > does -i grout:dpdp/selector then it loads libpcap-dpdp-plugin.so, and 
> > passes "selector" to it?
>
> Yes, that would be the idea.
>
> > The alternative is that tcpdump has a --load-plugins dpdk,vpp ... and 
> > if that shows up before --list-interfaces, then it enumerates the 
> > devices from those plugins?
> >
> > I would prefer --list-interfaces *NOT* attempt to load all the plugins 
> > in the default directory, and ask them for a list :-)
>
> That is up for discussion. Your proposal looks reasonable enough. Never 
> load any plugin *implicitly*.
>
> > > - ABI. Plugins must not see pcap-int.h. The proposal is a versioned, 
> > >   narrow accessor API (pcap_plugin_*) so libpcap internals stay 
> > >   private and the plugin ABI is explicit and stable.
> >
> > Agreed.
> > No public structures ideally, only getters/setters?
>
> Yes, pcap_t should be an opaque forward definition.
>
> > > - Failure behaviour. A plugin that is missing, unloadable, or reports 
> > >   a mismatched ABI version must be a silent no-op, never a fatal 
> > >   error for libpcap or its callers.
> >
> > I agree that it shouldn't call exit(2)!
> > I also agree that error strings need to be returned, not fprintf(stderr,..).
> > I'm not understanding the no-op :-)
>
> The plugin loader could get an error back and optionally log it via 
> strerror(). But since applications using libpcap are supposed to not 
> care about plugins, they should not be affected if one of them is 
> broken.
>
> > > 4. The attack surface
> > > =====================
> > >
> > > This is the part that most needs the group's judgement, and we do not 
> > > want to hand-wave it.
> > >
> > > The honest position is that dlopen()ing code into a process that may 
> > > hold capture privileges increases the attack surface, and the 
> > > framework must be built so that the increase is as close to zero as 
> > > possible.
> >
> > Thanks.
> > I'd like _tcpdump_ to drop any privilege if it is told to load 
> > a plugin.
> >
> > While no specific local kernel privilege is required to talk to DPDK, 
> > I would assume that the API into things requires something... group 
> > permissions to open a socket in a magic directory?
> > That kind of thing can just be an attribute of the user invoking 
> > things?
>
> Yes, we could think of a new flag for pcap_init():
>
>         #define PCAP_ENABLE_PLUGINS 0x00000004U
>
> To be explicitly passed by applications. E.g. for tcpdump, it could 
> translate to a new CLI flag and/or be enabled by default.
>
> > > - Off by default. The loader should be a compile-time option, 
> > >   disabled by default, as fxlb already requested on the PR. 
> > >   Distributions and dataplane projects opt in; a stock libpcap build 
> > >   carries none of it.
> >
> > Given this, does the plugin actually need to be loadable?
>
> What do you mean?
>
> > The problem with the existing contribution was the lack of 
> > maintenance/maintainers.  We needed private headers to even attempt to 
> > compile. If we don't compile it regularly, then we won't really know 
> > if it is broken. If the can stabilize an API (not ABI), then maybe 
> > that's the simpler way.
>
> Our proposal is to push the maintenance *outside* of libpcap. If the 
> API/ABI changes, then the maintainers of third-party plugins will need 
> to adjust their code.
>
> But then, as I stated earlier, the API should be minimal and simple 
> enough that it will not need to change often (if ever).
>
> > Alternatively: fork libpcap, rip EVERYTHING else out ("git rm", not 
> > edits). Leave that as one or two commits with the right code, and just 
> > rebase that on libpcap.git. Maybe we'd even accept that as 
> > a continuing branch.
>
> I'll take that as a trolling attempt :)
>
> Obviously, the goal of my email was to *avoid* forking. Besides, I don't 
> think my employer would be thrilled if we did!
>
> > > 5. Maintenance burden
> > > =====================
> > >
> > > To Denis's concern that a framework nobody uses, or one whose plugins 
> > > become TTG's responsibility, would be an expensive mistake: we agree, 
> > > and we think the framework should be designed so that no plugin is 
> > > ever TTG's responsibility. Plugins live in and are maintained by 
> > > their own projects (grout being a concrete, active example). Inside 
> > > libpcap the recurring cost is just the loader and the accessor ABI. 
> > > If we keep the ABI narrow and the loader off by default, that cost 
> > > stays small and self-contained, and there are no repository-boundary 
> > > problems inside libpcap itself.
> >
> > The best way to amortize and test the cost of the ABI is for libpcap 
> > to use that ABI to do, for instance TPACKET_V3.   This might let us 
> > remove some of the TPACKET3 bug-work around code by having an "older" 
> > plugin. We have lots of #ifdef HAVE_LIBNL around nl80211 code, and 
> > maybe that could be a plugin.  Maybe we could drop TPACKET_V2, 
> > although that might not be much of a win.
>
> That rejoins what Bill was proposing: change builtin backends to use 
> that plugin API. That way, it will be tested.
>
> > Thank you for the engaging email!
>
> Thanks for your very detailed response! Hopefully we're going in the 
> right direction ;)
>
> --
> Robin
>
> # Made with real ingredients.
>
>
> --- End Message ---
_______________________________________________
tcpdump-workers mailing list -- tcpdump-workers () lists tcpdump org
To unsubscribe send an email to tcpdump-workers-leave () lists tcpdump org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s