Re: Proposed new pcap format

[Jefferson Ogata <Jefferson.Ogata () noaa gov>]

Christian Kreibich wrote:
> Are you suggesting an xml-based pcap, or xmlified tcpdump output?
>
> If you mean the former, I think the problem with this approach is that
> in order to be able to write out a file in the first place, the
> structure of the packet content has to be understood by libpcap (so that
> it knows to write "<ip vers='4' hlen='20' ... flags='0x04' ...
> proto='17'>" etc) -- then the question becomes what to do with unknown
> protocols etc.

I'm suggesting the pcap storage format be XML. A raw capture, without using 
protocol dissectors, would just be a sequence of base64-encoded (perhaps) frames 
and metadata.

Tools like the tcpdump protocol dissectors and tethereal could then just be XML 
filters that take a raw XML input frame and annotate it with protocol elements, 
as in the rough example I posted. Existing XML tools, e.g. xsltproc, could 
generate reports from the annotated XML using XSLT. The reports could as easily 
be HTML output as plain text or more XML.

Additional protocol dissectors for protocols unknown to tcpdump/tethereal could 
be written in any language with XML support (preferably event-based). In fact, 
many protocol analyzers could be written directly in XSLT/XPath and processed 
using xsltproc. Among other things, this provides many means to eliminate the 
continuing problem of buffer overflows. tcpdump could have a plugin architecture 
with an XML filter for each protocol/frame type.

> I think what you're proposing should be provided by an xmlified tcpdump,
> but not the capture library.

I'm suggesting that we use XML as the capture file format so that tcpdump 
becomes an extensible XML filter.

Or you can throw all that musing away. Just pay attention to the discussion for 
a little while -- it revolves around timestamp and metadata formats, sizes of 
fields, and other esoterica that are sounding a bit archaic in today's computing 
environment. I think we should take a hard look at whether it's really 
appropriate to define yet another hard binary file format when XML can provide 
the same functionality with modest storage overhead, and has many added benefits.

--
Jefferson Ogata <Jefferson.Ogata () noaa gov>
NOAA Computer Incident Response Team (N-CIRT) <ncirt () noaa gov>

-
This is the tcpdump-workers list.
Visit https://lists.sandelman.ca/ to unsubscribe.