Home page logo
/

nmap-dev logo Nmap Development mailing list archives

Re: [RFC][patch] XML structured script output (summary of output changes)
From: David Fifield <david () bamsoftware com>
Date: Wed, 13 Jun 2012 19:33:58 -0700

On Mon, May 21, 2012 at 08:57:38PM -0500, Daniel Miller wrote:
Congratulations everyone on version 6!

For a year or so, I've been looking for feedback on my patch for XML
structured output for NSE scripts [1], and as it has been a while, I
thought I'd bring it up again. I've fixed a couple bugs since last
time, and I've started keeping my local copy of the Nmap source under
Git source control, so I can rebase my branch as development continues
on the official source. I'm attaching 2 patches: nse-structured.diff
has all the code changes to use XML output for scripts, and
scripts-structured.diff patches a few scripts to take better advantage
of the structured output.

As far as impact to script writers, there shouldn't be much, since a
table passed to stdnse.format_output() will work as it always has, as
will simply returning a string from a script. In this regard, the
functionality is largely opt-in. If the patch is accepted, I'll spend
time updating existing scripts to structure their output.

There is room for improvement: Currently, "horizontal lists" don't
work; that is, comma-separated values on one line of normal output
should be output as a list of <elem> elements in XML. The base
functionality is there, with ScriptOutputNode.set_display(FLOW), but
the logic to tie it in as a Lua function is missing. I don't figure
this is a big deal, and can be added in later.

Thank you for following this so persistently, Dan. I think we are close
to finally getting something merged. There is a lot to discuss so I'm
going to write several messages. This one is a summary of the output
changes made by your patch.

As I understand the patch, it co-opts stdnse.format_output and uses it
as the mechanism to output structured XML. The XML structure will mirror
that of the table given to stdnse.format_output. However, scripts that
use format_output merely as a way to output some nested lines, need to
be modified to break their output into more structured name-value pairs,
otherwise the structured XML output is essentially the same sequence of
lines.

Here is normal screen output with some typical script results.

# nmap -6 -n -Pn -p443 --script=address-info,ssl-cert fe80::1e6f:65ff:feb6:cad3 ::1
Nmap scan report for fe80::1e6f:65ff:feb6:cad3
Host script results:
| address-info: IPv6 EUI-64
|_  MAC address: 1c:6f:65:b6:ca:d3 (Giga-byte Technology Co.)
Nmap scan report for ::1
PORT    STATE SERVICE
443/tcp open  https
| ssl-cert:
|   Subject: commonName=localhost
|   Issuer: commonName=localhost
|   Public Key type: rsa
|   Public Key bits: 1024
|   Not valid before: 2012-06-14 01:05:29
|   Not valid after: 2013-06-14 01:05:29
|   MD5: 62b8 3671 9c3c fb52 e3e9 4db3 254d d50c
|_  SHA-1: 7514 8e75 f165 d8b5 7cbc 72ae f534 22e1 eb8d d0f4

This is what the XML output looks like now. Script output is stuffed as
a string in //script () output elements.

    <hostscript>
      <script id="address-info" output="IPv6 EUI-64; MAC address: 1c:6f:65:b6:ca:d3 (Giga-byte Technology Co.)"/>
    </hostscript>
    <ports>
      <port protocol="tcp" portid="443">
        <script id="ssl-cert" output="Subject: commonName=localhost&#xa;Issuer: commonName=localhost&#xa;Public Key 
type: rsa&#xa;Public Key bits: 1024&#xa;Not valid before: 2012-06-14 01:05:29&#xa;Not valid after:  2013-06-14 
01:05:29&#xa;MD5:   62b8 3671 9c3c fb52 e3e9 4db3 254d d50c&#xa;SHA-1: 7514 8e75 f165 d8b5 7cbc 72ae f534 22e1 eb8d 
d0f4"/>
      </port>
    </ports>

Here is the XML output using your nse-structured3 patch, without any
script changes. The only change is that output moves from the
//script () output attribute to //script/elem elements.

    <hostscript>
      <script id="address-info">
        <elem>IPv6 EUI-64; MAC address: 1c:6f:65:b6:ca:d3 (Giga-byte Technology Co.)</elem>
      </script>
    </hostscript>
    ...
    <ports>
      <port protocol="tcp" portid="443">
        <script id="ssl-cert">
          <elem>Subject: commonName=localhost
Issuer: commonName=localhost
Public Key type: rsa
Public Key bits: 1024
Not valid before: 2012-06-14 01:05:29
Not valid after: 2013-06-14 01:05:29
MD5: 62b8 3671 9c3c fb52 e3e9 4db3 254d d50c
SHA-1: 7514 8e75 f165 d8b5 7cbc 72ae f534 22e1 eb8d d0f4</elem>
        </script>
      </port>
    </ports>

Here is the XML script output after making format_output changes to
certain scripts. The big change here is that "Label: value" data now
appear as <elem key="Label">value</elem>. Also, multiple lines from
format_output get their own //script/elem elements.

    <hostscript>
      <script id="address-info">
        <container name="IPv6 EUI-64">
          <elem key="MAC address">1c:6f:65:b6:ca:d3 (Giga-byte Technology Co.)</elem>
        </container>
      </script>
    </hostscript>
    ...
    <ports>
      <port protocol="tcp" portid="443">
        <script id="ssl-cert">
          <elem key="Subject">commonName=localhost</elem>
          <elem key="Issuer">commonName=localhost</elem>
          <elem key="Public Key type">rsa</elem>
          <elem key="Public Key bits">1024</elem>
          <elem key="Not valid before">2012-06-14 01:05:29</elem>
          <elem key="Not valid after">2013-06-14 01:05:29</elem>
          <elem key="MD5">62b8 3671 9c3c fb52 e3e9 4db3 254d d50c</elem>
          <elem key="SHA-1">7514 8e75 f165 d8b5 7cbc 72ae f534 22e1 eb8d d0f4</elem>
        </script>
      </port>

David Fifield
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/


  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]
AlienVault