Re: RFC: a loadable plugin framework for external capture backends

[Michael Richardson <mcr () sandelman ca>]

Guy Harris <gharris () sonic net> wrote:
    > On Oct 1, 2026, at 1:33 PM, Michael Richardson <mcr () sandelman ca> wrote:

    >> https://adam.pages.nic.cz/dns-probe/Running.html does not, AFAIK,
    >> use libpcap for these preformance reasons, offering TPACKET or DPDK integrations.

    > libpcap runs atop TPACKET on Linux, so presumably the performance
    > problem are either in pcap.c or pcap-linux.c. Any idea what they might
    > be?

I think that they wanted more detailed access to the packets, they wanted to
be able to use multiple threads.  Like, an initial classifier that looks for
5-tuples, then passes that packet to a thread that deals with that particular
flow.   I think, that this would permit the different threads to be able
to release the buffers back to the kernel asynchronously to each other.
(When dns-probe was in progress, I was actually working on a similiar project
for another ccTLD, and once dns-probe was announced... we stopped)

    >> I think that the rpcap protocol *itself* could be better, mind you.

    > Yes, that's why I added versioning support (via some hackery to allow
    > version information to be exchanged without breaking older client or
    > server implementations).

Good.

    >> 2. some systems (distros) use capabilities, set on the tcpdump executable.

    > Better, if you *have* those capabilities (that's a Linuxism) and they
    > suffice for capturing on the device in question (which is, alas, *not*
    > true for capturing on USB buses, for example).

Yes, annoying that different sources require different things, and the
complexity of having an administrator set this up for a "mortal" user
properly often leads to... well (1) instead.

    >> 3. but, many of us prefer to just sudo.

    > OK, I guess.  (I just rely on Wireshark's ChmodBPF to make all the BPF
    > devices usable.)

:-)

    > Personally, I like the idea of:

    > tcpdump (or TShark or Wireshark or...) is a front-end program;

    > the dissectors run in a separate, possibly sandboxed, process, with the
    > front-end doing whatever file opening is necessary;

Exactly, it would be ideal.
There is a fair bit of stuff to coordinate when trying to roll file names,
overwritting the oldest files, etc.

    >>> - Discovery. Where are plugins found, and in what order? (The PR uses    a
    >>> single compiled-in directory plus an environment override that is
    >>> dropped under elevated privileges, like ld.so with LD_LIBRARY_PATH.)
    >>
    >> Yes, and that's where I begin to get afraid, because features were added to
    >> ld.so, and sometimes they interacted in curious and insecure ways.
    >> There may be *ld.so* options that affect us even.

    > Plus, ld.so isn't a single program, it's several programs that do
    > similar things with ELF files, plus there's one called just "dyld" that
    > deals with Mach-O files, and then there's what mechanism Windows uses
    > for dynamic loading of libraries and plugins, so it's not as if
    > "understanding what's safe" is necessarily a problem that has a
    > universal solution for all platforms libpcap supports.

You've told me of complexities in other things that I did not know, and I'm
now even more alarmed that the set of interactions is unknowable going forward.

    >> Further, all of this probably has to work when it's rpcapd (under wireshark,
    >> from a windows desktop user...) that is doing this.
    >> That's where having multiple plugins loaded might be required.
    >>
    >> *The rpcap client becomes a plugin*

    > The rpcap should, at minimum, plug into a similar interface as the ones
    > that local capture modules already do. That way, other over-the-wire
    > capture mechanisms could be supported.

+1.
This also would isolate all that code behind a different release cycle and
vulnerability process.

    > Making them plugins allows Somebody Else to support them (or, in the
    > case of rpcap, allows a separate project to handle both the rpcap:
    > handler and the rpcapd daemon, even if it's a Tcpdump Group project).

I know how we got rpcap, but I dislike that it wasn't just "ssh host rpcap".
I know... popen("ssh..") is only just becoming common on windows, and that
was just hard to very difficult ten+ years ago.

I would revise things so that the data/packet flow was on a different port,
encrypted seperately from the SSH.  Like how "mosh" works.
Probably using UDP.  Maybe QUIC. (Ideally: SCTP).
It would be nice if windows could use kerberos too.

For a lot of situations, one is reaching out to some "cloud" device, or
across a NOC network (like an RFC8994 ACP...), so if done right, pinholes can
be opened for the desktop system to receive packets.

Again, probably not going to handle a full mirror port from 100G forwarding
engine... but if the BPF program is only selecting a small number of packets,
it becomes way more workable for many.
Clearly, "rpcapd" using libpcap can use "plugins".

    >> I'm sure that there are programs that use more than one pcap_t, and quite
    >> possibly they might need to have different plugins.

    > Wireshark's dumpcap (which does all the capturing for Wireshark and
    > TShark) can capture on multiple devices, so it can have more than one
    > pcap_t open.

So, not one dumpcap per device, but one dumpcap per wireshark instance.

    > Once libpcap has full pcapng support, tcpdump should probably allow
    > multiple -i options, just as TShark and dumpcap do.

Agreed.
Although, I think might be a win if one actually captures in multiple
processes, writes to multiple files, and then combine them afterwards.

Still, a thing I'd like a tool that looks at multiple interfaces on a router
(like the PPPoE + "br-lan" on an openwrt), and then logs/notes packets that
do *not* appear on both interfaces :-)

    >> I'd like _tcpdump_ to drop any privilege if it is told to load a plugin.

    > I'd ideally like *tcpdump* not to run with any privileges, other than
    > what privileges it gets by having a descriptor handed to it - or,
    > rather, to libpcap - by a small simple program that runs with elevated
    > privileges (see above).

+1

    >>> - Off by default. The loader should be a compile-time option, disabled    by
    >>> default, as fxlb already requested on the PR. Distributions and
    >>> dataplane projects opt in; a stock libpcap build carries none of it.
    >>
    >> Given this, does the plugin actually need to be loadable?

    > So you're saying that plugins should be bundled *by the OS distributor*
    > and wired in, rather than allowing outside-of-libpcap projects *not*
    > bundled by the distributor to be plugged into libpcap?

I'm saying that calling dlopen() shouldn't be required: do that at link time.

    > If so, that might work for most cases with Linux distributions, most of
    > which don't seem to draw a strong distinction between "core OS" and
    > "packages". It *does* mean that the libpcap package would have more
    > dependencies.

Yes, that's the problem with "compile-time option, disabled"..

    > It might not work so well for the *BSDs, most of which have a "core OS"
    > separate from the package collection, and that include libpcap in the
    > "core OS". That includes macOS ("CupertinoBSD"), which, being a
    > commercial OS, may have third-party *binary-only* libpcap extensions,
    > which would require a stadardized *ABI*. (macOS is one reason why I
    > try to make sure libpcap never changes its ABI other than, perhaps,
    > carefully dropping routines from the API/ABI over time, as long as
    > they've been deprecated before their disappearance.)

libpcap 2.x

    >>> - Trusted location only. Plugins load only from a root-owned,
    >>> non-user-writable directory; the environment override is ignored    under
    >>> elevated privileges, exactly as ld.so treats LD_LIBRARY_PATH.
    >>
    >> Well, if libpcap drops any privileges it might have had the moment
    >> --load-plugin is seen (but, before acting on it), then actually, this is moot.

    > I'm not sure *libraries* should drop privileges on their own, as the
    > program may be running with elevated privileges for other reasons.

Yeah, I mispoke:

      } Well, if {tcpdump, tshark, dumpcap} drops any privileges it might
      } have had the moment --load-plugin is seen (but, before acting on it),
      } then actually, this is moot.

    >> The best way to amortize and test the cost of the ABI is for libpcap to use
    >> that ABI to do, for instance TPACKET_V3.   This might let us remove some of
    >> the TPACKET3 bug-work around code by having an "older" plugin.
    >> We have lots of #ifdef HAVE_LIBNL around nl80211 code, and maybe that could
    >> be a plugin.  Maybe we could drop TPACKET_V2, although that might not be much
    >> of a win.

    > Currently we use TPACKET_V2 for immediate mode, as I don't think
    > there's a way to get "give me a packet as soon as it arrives, no
    > batching" from TPACKET_V3.

cp pcap-linux.c pcap-linux-tpacketv2.c
mv pcap-linux.c pcap-linux-tpacketv3.c
.. edit until they are two things, both internal "plugins"

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        | network architect  [
]     mcr () sandelman ca  http://www.sandelman.ca/        |   ruby on rails    [



_______________________________________________
tcpdump-workers mailing list -- tcpdump-workers () lists tcpdump org
To unsubscribe send an email to tcpdump-workers-leave () lists tcpdump org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s