Re: RFC: a loadable plugin framework for external capture backends

[Denis Ovsienko <denis () ovsienko info>]

On Tue, 13 Oct 2026 12:41:53 +0200
"Robin Jarry" <rjarry () redhat com> wrote:

> Denis Ovsienko, Oct 12, 2026 at 15:28:
> > Let's not over-engineer this aspect in a rush.  A shared ring
> > buffer could be a protocol option and the mandatory baseline could
> > multiplex everything over exactly one bi-directional channel.  Not
> > all programming languages expose all OS APIs, and it seems likely
> > that a plug-in developer would like to keep things as simple as
> > possible in the initial proof-of-concept implementation.  
>
> I agree that we must not assume a shared ring buffer is always 
> available. But then, if we exchange packets inline through the
> socket, isn't that reinventing RPCAP?

Not quite exactly: rpcap was inherited as-is, and a plug-in protocol
can be designed more or less differently based on the lessons learned so
far.  For example, in rpcap it is wrong to compile the filter at the
user end of the connection, but in a plug-in protocol it may be a
better design choice.

> I don't mind it particularly as long as a zero system call ring
> buffer is included from day one.
>
> > One of the typical reasons people capture packets is to debug 
> > functional behaviour rather than performance.  In other words,
> > often to debug a subject matter problem it is sufficient to
> > establish that what was supposed to be an exchange of 10 packets
> > has terminated after 5 packets or never started in the first place
> > or a network address was incorrect.  Capturing at many Mpps is a
> > separate use case, which some applications just never have
> > (consider DLT_RDS, DLT_MDB, DLT_LORATAP and the likes).  
>
> I don't think libpcap should be used for high rate packet mirroring
> on disk. On the contrary, I completely agree with you that this is 
> a *debugging* tool.
>
> But the fact that your capture endpoint (e.g. tcpdump) is slow does
> not mean it should slow down the datapath. I would prefer if we could
> keep the high performance requirements *on the producer side* in 
> consideration.

The receiver can be slower than the sender for any reason and for any
length of time.  This problem is not new in libpcap and can use the same
approach as with regular network interfaces: absorb short bursts in a
buffer, on a full buffer drop captured packets and increment the drop
counter.  Report the statistics accurately and let the end user decide
what to do about any drops.

> > It is not a matter of preference, but a matter of how well a
> > solution space can relate with the problem space.  To that end,
> > Netlink is Linux-specific, it would add a library dependency and
> > would likely overshoot the problem space, so it would need to be
> > something else.  
>
> Understood. In that case, what do you think if we keep the *control* 
> protocol as simple as possible? Without any serialization overhead.
> Just (packed?) C structures with a well-known preamble that carries
> the protocol version.
>
> This can be adapted to virtually any programming language.

The plug-in protocol would indeed need to be versioned.  The very first
version would need to include what is essential only.  Guy is most
familiar with rpcap encoding, perhaps he could comment which design
elements could be reused and which should be avoided?

> The shared ring buffer part is a bit more specialized and would only 
> work with low level languages as long as they comply to the official 
> memory layout.

-- 
    Denis Ovsienko
_______________________________________________
tcpdump-workers mailing list -- tcpdump-workers () lists tcpdump org
To unsubscribe send an email to tcpdump-workers-leave () lists tcpdump org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s