Re: RFC: a loadable plugin framework for external capture backends

["Avinash H. Duduskar" <avinash.duduskar () gmail com>]

Michael Richardson <mcr () sandelman ca> wrote:
> So, let me suggest different interfaces which are relatively stable, and are
> already shared memory based: TPACKET_V3 and AF_XDP:

One correction on AF_XDP, from having built a libpcap backend on it. Vadim
covered the DPDK half; this is the other half.

An AF_XDP socket is not a tap. The XDP program hands a frame to the socket
with bpf_redirect_map(), and that consumes it; the stack never sees it.
(BPF_F_BROADCAST clones between devices in a devmap; nothing clones into
an XSKMAP while passing the original up.) A capture on queue N of eth0
takes queue N away from the host while it runs. Guy asked in October 2019
whether it can deliver copies of packets that also reach the stack [1];
the answer is still no. So an AF_XDP backend sits in the same class as
netmap: and the removed dpdk:, a prefix device with takeover semantics,
outside the default path, and ingress only: "-i xdp:eth0" sees nothing
the host sends.

> There is a win here in that libpcap could wind up with an AF_XDP capture
> path.  Obviously, the setsockopt() calls that AF_XDP requires have to
> replaced with a different communication.  Probably a UNIX domain socket,

That is a different thing. AF_XDP is a capture source on a kernel-driven
NIC; what you describe is a transport between a privileged capture
process and an unprivileged front end, which could carry frames from any
source. Complementary, not the same interface.

For the record, since the thread is deciding what the capture API should
look like: I have a working AF_XDP backend, an "xdp:ethN[:qN]" prefix
device, copy mode, single queue, userspace cBPF filter, RX only, on
libxdp's default redirect program. It builds clean on current master with
the full warning set (CMake gcc and clang, autoconf). A veth suite run
on 2 October against that build measures:

- AF_PACKET on the bound queue goes dark for the life of the capture
  (40 of 40 frames diverted); the capture sees only its own queue (52 of
  192 flows over four queues). Takeover is per queue.
- Egress is never seen; XDP is an RX hook.
- Privileges are all spent at activate; the receive loop makes no
  privileged call. On Linux 7.2 with libxdp 1.6.3 the smallest set that
  activates is {NET_RAW, NET_ADMIN, SYS_ADMIN}. SYS_ADMIN is the
  dispatcher's cost, not the interface's: its setup needs a lookup only
  SYS_ADMIN grants (PERFMON does not substitute), while the same redirect
  program attached directly binds with {NET_RAW, NET_ADMIN, BPF}. The
  per-set results are in the harness on the branch.
- Link down is silent; link delete is detected through SO_ERROR.
- A SIGKILL'd capture leaves only a stray XDP program attached. Host
  traffic is back by the first probe a second later with no redirects
  firing, and the next capture rebinds in about a quarter of a second,
  detached or not (two runs, June and 2 October).

Caveats: veth only, no physical NIC yet. Copy mode, so no speed claim;
Robin's point stands. The one library dependency is libxdp, for its
dispatcher so the capture program coexists with other XDP programs. That
is a choice, and it is what costs CAP_SYS_ADMIN above; the kernel
interface itself does not.

Two observations. The privilege shape fits what Guy described: a small
helper could create the socket and UMEM, bind, attach the program and
own the bpffs pin, then pass the socket fd and a memfd for the UMEM to a
process with no capabilities that never touches bpffs. With the
dispatcher that helper is root in all but name, which is the case for
keeping it apart from the reader.

Branch, with the test harness:
https://github.com/Strykar/libpcap/tree/afxdp-spike

[1] https://www.mail-archive.com/tcpdump-workers () lists tcpdump org/msg08857.html


- Avinash
_______________________________________________
tcpdump-workers mailing list -- tcpdump-workers () lists tcpdump org
To unsubscribe send an email to tcpdump-workers-leave () lists tcpdump org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s