BPF issues that require careful planning

[Denis Ovsienko <denis () ovsienko info>]

Hello all.

There exist a number of imperfections in BPF syntax and implementation,
which have little in common except addressing them would change one or
another long-established behaviour.

1. The long-standing VLAN/MPLS implicit offset shift, which has been
   discussed in great detail already.  A potential way to disambiguate
   such keywords could be replacing each of these with two new keywords,
   for example, "vlan-pop" (the current behaviour) and "vlan-peek"
   (the version that leaves the offset intact).  This could be done in
   two steps: after introducing the new keywords the old keywords would
   still work, but generate a warning in pcap_compile(), then after a
   while the old keywords would become invalid.

2. Protocol name to number resolution sometimes uses the libpcap's own
   IPPROTO_xxxxx constants, sometimes it uses the OS IPPROTO_xxxxx
   constants and sometimes it uses getprotobyname(), which in most libc
   implementations uses either /etc/protocols (or the equivalent file)
   or a network database.  Except musl libc, which hard-codes the
   protocol list.  These compile-time and run-time differences can
   cause the same filter expression to fail to compile (e.g. "proto
   \sctp" with musl libc, as discussed in libpcap pull request #1399)
   or to compile to a different bytecode (e.g. "igrp" on FreeBSD, see
   libpcap pull request #1428).  A potential solution is to establish an
   internal registry of well-known protocols and to try it either
   before or after (likely not instead) the traditional means of
   protocol lookup.

3. The filter syntax among many other elements includes "byte", which
   looks a forgotten early draft of the index operation, the binary
   logic operators and the bitwise operators:
   * "byte X = Y" is the same as "link[X] = Y"
   * "byte X < Y" is the same as "link[X] < Y"
   * "byte X > Y" is the same as "link[X] > Y"
   * "byte X | Y" and "byte X & Y" do not use the X, and instead test
     a function of Y and the current value of the accumulator without
     setting the accumulator first
   Both "byte" and the more developed elements are present in tcpdump
   2.0, but "byte" was never documented.  It seems most reasonable to
   assume "byte" a never used keyword and to replace it with a short
   note in the backward compatibility section of pcap-filter(7), but
   this would still be a change of a behaviour that exists since at
   least 1991, so it demands a certain order.

4. The "enable/disable IPv6" build option, when non-standard IPv6 stacks
   were commonplace and libpcap depended on these, had a purpose and
   managed the optional dependency.  After quite some development in
   libpcap and elsewhere --enable-ipv6 has been the default for a while
   and the effect of --disable-ipv6 became very patchy.  For example,
   "ether proto \ip6" stops working, but "ip6" continues to work, even
   though both mean exactly the same (EtherType 0x86dd) and do not
   depend on IPv6-specific code.  Likewise, "ip6 host ::1" and "ip6 net
   ff::/8" consistently stop working, but "icmp6", "ip6 proto \tcp",
   "ip6 and tcp port 80" and "ip6 protochain 17" continue to work.  It
   seems most useful to lose the build option and to make all IPv6
   syntax always available.  (This would also eliminate a half of
   libpcap CI build matrix.)

5. On this note, I guess that around 99% of libpcap users no longer
   encounter live or recorded DEC (including DECnet) protocols, ATM,
   Token Ring and FDDI traffic.  Yet the corresponding elements of BPF
   syntax are not conditional, as IPv6 aims to be.  This was a
   reasonable approach in the past, but for present-day networks it
   would make more sense to make it the other way around and to make
   antique protocols _filtering_ support optional (whether opt-in or
   opt-out).

6. In the long-undocumented MTP2 department the "lssu" keyword is an
   alias for "lsu", which originally was a spelling error.  "lsu" became
   available at the same time as "lssu" (libpcap 0.9.6 in 2007), but
   neither was documented until very recently (pcap-filter(7) in the
   master branch now documents "lssu").  So it may be a good idea to
   replace "lsu" with a backward compatibility note.

7. The semantics of "host A" is the same as that of inet_aton(), but the
   semantics of "host A.B" and "host A.B.C" is the same as that of "net
   A.B" and "net A.B.C" (see my earlier message with the subject "IPv4
   address format ambiguity").  Besides the obvious need to document
   the implemented behaviour, a potential way to make the syntax a bit
   more consistent could be parsing A by "net A" rules when A <= 255
   and parsing it by inet_aton() rules in all other cases.

There are likely other similar issues that would not justify a
backward-incompatible change on their own.  However, coalescing several
such changes as a feature of a major libpcap release would minimise the
inconvenience of the upgrade (it is expected that major releases can
make backward-incompatible changes, if appropriately documented), would
improve the SNR in the code base and documentation, and would reduce
the attack surface.  To that end, I had approximately the following
approach in mind:

1. The current master branch (to-be-1.11.x) already includes a number of
   changes that remove support for particular OSes and capture modules.
   However, it does not make any changes to the BPF syntax except bug
   fixes (e.g. DECnet parsing) and more accurate documentation.  Let's
   document as many known imperfections as possible before making any
   changes, and try to cover the complete syntax with unit tests.

2. If everything goes well, the 1.11.0 release will have the
   traditional BPF syntax and better documentation.

3. After making the libpcap 1.11.0 release and addressing any immediate
   breakage the master branch will become to-be-libpcap-2.0.  This will
   be the testing ground for any substantial changes, then things that
   work well can be ported to 1.11.x to provide a migration path.  For
   example, in 1.11.x eventually there would be an opt-out for antique
   protocols and new-style keywords for VLAN (with a PCAP_WARN_xxxxx in
   case a filter uses the old-style VLAN keyword), but otherwise things
   by default would work as before.  Let's suppose the 1.11.x series
   lasts a few years and provides a sufficient migration window even
   after libpcap 2.0.0 has stabilised.

4. In the master branch the to-be-libpcap-2.0 would look more like
   libpcap 2.0.0 will eventually look: no support (or opt-in) for the
   old-style VLAN keywords and opt-in for antique protocols.

What could possibly go wrong?

-- 
    Denis Ovsienko
_______________________________________________
tcpdump-workers mailing list -- tcpdump-workers () lists tcpdump org
To unsubscribe send an email to tcpdump-workers-leave () lists tcpdump org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s