Re: RFC: a loadable plugin framework for external capture backends

[Vadim Goncharov <vadimnuclight () gmail com>]

On Mon, 12 Oct 2026 14:23:15 +0200
Robin Jarry via tcpdump-workers <tcpdump-workers () lists tcpdump org> wrote:

> Hi Vadim,
>
> Vadim Goncharov, Oct 12, 2026 at 13:02:
> > > As I already said, sending the packets in a socket (UNIX or 
> > > otherwise) is not an option because it requires system calls. We 
> > > "could" run an intermediate processes that reads from a shared memory 
> > > ring and copies to a socket but that seems extraneous and we'd prefer 
> > > to reduce the number of moving parts.
> > >
> > > However, I don't know if sharing an anonymous ring-buffer [1] in 
> > > memory is possible with all backends that libpcap supports (is there 
> > > file descriptor sharing over SCM_RIGHTS [2] on windows?). There could 
> > > be different options to allow sharing over POSIX shared memory 
> > > segments [3].
> > >
> > > [1] https://www.mankier.com/2/memfd_create
> > > [2] https://www.mankier.com/7/unix#Description-Ancillary_messages  
> >
> > No, Windows uses it's own mechanisms, Denis' proposal was conceptual 
> > ("socket" as "bi-directional communication channel", not as "UNIX 
> > socket") rather than exact technology, but why SCM_RIGHTS is needed? 
> > It could be e.g. just opening a named pipe (and Windows has named 
> > pipes, but not in filesystem).  
>
> The named pipe can solve the *control* part. But what mechanism would 
> you use to exchange the *packets* without any system call on Windows? Is 
> there a way to expose a memory segment from one process to another like 
> shm_open() or memfd_create()?

Yes, according to Google, the primary functions used are:

    CreateFileMapping(): Creates a file mapping object.
        If you pass INVALID_HANDLE_VALUE instead of a real file handle, Windows backs the shared memory using the 
system paging file (making it anonymous/non-persisted, similar to memfd_create()).
        You can assign it a string name (making it a named shared memory object, similar to shm_open()) or leave it 
unnamed and share the handle via inheritance or handle duplication. 

MapViewOfFile(): Maps a view of the file mapping object into the address space of the process, returning a usable 
pointer. 
OpenFileMapping(): Allows a second process to open an existing named mapping by its string name. 
DuplicateHandle(): If the mapping is unnamed, the creator process can duplicate the internal OS handle and pass it to a 
target process via another IPC channel (like a named pipe or command-line argument). 

> > > [3] https://www.mankier.com/3p/shm_open
> > >
> > > As for the control API, anything is on the table. Whatever will be 
> > > simpler for libpcap. Would you prefer an ad-hoc binary message 
> > > protocol   
> >
> > Instead of reinventing ad-hoc wheel, CBOR [RFC 8949] is a good fit 
> > - it is:
> > * standard [STD 94]
> > * simple (just few hundreds line of C, or less if you do not need 
> >   general interoperability for partial implementation)
> > * and yet extensible (can be described in two words as "binary JSON 
> >   with extensions").  
>
> I don't mind about using CBOR. But that does not solve the issue of the 
> *structure/schema* of the exchanged messages. As far as I can tell, CBOR 
> only offers encoding/decoding. But you have to build your own schema 
> validation and API versioning on top of it.
>
> Unless we are 100% certain that the API will never change, it is hard to 
> decide which encoding will be best.

Exactly because of that, CBOR fits better due to extensibility. Schema
validation and versioning are much easier tasks when you can, for example,
just add a new key to a map, which is ignored by other side - similar to as it
happens in JSON/Web world.

> More importantly, our requirement for zero system calls is crucial. That 
> would defeat any encoding (binary or otherwise) for packet exchange 
> between the backend and libpcap. The only viable solution is a lockless 
> ring in shared memory using stdatomics for synchronization.
>
> To avoid extra complication, I am tempted at imposing a C header type 
> API contract where all control messages and the ring buffer are 
> described as standard C structures along with static inline helpers for 
> inserting/popping packets to/from the ring buffer.

For actual highload data path, extensibility requirement is relaxed, so using
fixed C structs may be reasonable. Hovewer, don't forget history with .pcap
and .pcapng - published in 2005 and then newer in already 2009.

> There could be non-static inline functions to handle the control message 
> exchanges in libpcap.so. The backend could link against the library to 
> parse a control message and respond accordingly.
>
> Does that sound reasonable?

If we go the "data only" protocol, why to link anything?

-- 
WBR, @nuclight
_______________________________________________
tcpdump-workers mailing list -- tcpdump-workers () lists tcpdump org
To unsubscribe send an email to tcpdump-workers-leave () lists tcpdump org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s