Home page logo

nmap-dev logo Nmap Development mailing list archives

Re: [RFC][patch] XML structured script output
From: Daniel Miller <bonsaiviking () gmail com>
Date: Sun, 27 May 2012 22:19:46 -0500

I've attached 2 xml files, before.xml and after.xml, which I hope I've
sanitized sufficiently from a quick scan of my network (-sC). Note
that these represent the output of scripts that have not been modified
in any way from their current state. Modifying scripts to use the
format recommended by stdnse.format_output will result in more useful
organization of results in the new XML format, and further clarifying
key-value pairs when returning results will make results even more
useful. See the documentation in the patch for stdnse.format_output
for an example:

---Takes a table of output on the commandline and formats it for display to the
-- user. This is basically done by converting an array of nested tables into a
-- string. In addition to numbered array elements, each table can have a 'name'
-- and a 'warning' value. The 'name' will be displayed above the table, and
-- 'warning' will be displayed, with a 'WARNING' tag, if and only if debugging
-- is enabled.
-- Here's an example of a table:
-- <code>
--   local domains = {}
--   domains['name'] = "DOMAINS"
--   table.insert(domains, {['Domain 1']="example.com"})
--   table.insert(domains, {['Domain 2']="nmap.org"})
--   local names = {}
--   names['name'] = "NAMES"
--   names['warning'] = "Not all names could be determined!"
--   table.insert(names, "Name 1")
--   local response = {}
--   table.insert(response, "Apple pie")
--   table.insert(response, domains)
--   table.insert(response, names)
--   return stdnse.format_output(true, response)
-- </code>
-- With debugging enabled, this is the output:
-- <code>
--   Host script results:
--   |  smb-enum-domains:
--   |    Apple pie
--   |    DOMAINS
--   |      Domain 1: example.com
--   |      Domain 2: nmap.org
--   |    NAMES (WARNING: Not all names could be determined!)
--   |_     Name 1
-- </code>
-- The same table will also be rendered for XML output as
-- <code>
-- <script id="smb-enum-domains">
--   <elem>Apple pie</elem>
--   <container name="DOMAINS">
--     <elem key="Domain 1">example.com</elem>
--     <elem key="Domain 2">nmap.org</elem>
--   </container>
--   <container name="NAMES" warning="Not all names could be determined!">
--     <elem>Name 1</elem>
--   </container>
-- </script>
-- </code>
-- () param status A boolean value dictating whether or not the script succeeded.
--              If status is false, and debugging is enabled, 'ERROR'
is prepended
--              to the line. If status is false and debugging is
disabled, no output
--              occurs.
-- () param data   The table of output. This table may be modified by the function.
-- @return <code>nil</code>, if <code>data</code> is empty or status is false
--         and debugging is disabled, otherwise a table properly formatted
--         for the Scripting Engine to output.


On Sun, May 27, 2012 at 10:42 AM, Djalal Harouni <tixxdz () opendz org> wrote:
Hi Dan,

Thank you for re-sending, can you please send a sample XML file to show
the difference ?

On Mon, May 21, 2012 at 08:57:38PM -0500, Daniel Miller wrote:
Congratulations everyone on version 6!

For a year or so, I've been looking for feedback on my patch for XML
structured output for NSE scripts [1], and as it has been a while, I
thought I'd bring it up again. I've fixed a couple bugs since last
time, and I've started keeping my local copy of the Nmap source under
Git source control, so I can rebase my branch as development continues
on the official source. I'm attaching 2 patches: nse-structured.diff
has all the code changes to use XML output for scripts, and
scripts-structured.diff patches a few scripts to take better advantage
of the structured output.

As far as impact to script writers, there shouldn't be much, since a
table passed to stdnse.format_output() will work as it always has, as
will simply returning a string from a script. In this regard, the
functionality is largely opt-in. If the patch is accepted, I'll spend
time updating existing scripts to structure their output.

There is room for improvement: Currently, "horizontal lists" don't
work; that is, comma-separated values on one line of normal output
should be output as a list of <elem> elements in XML. The base
functionality is there, with ScriptOutputNode.set_display(FLOW), but
the logic to tie it in as a Lua function is missing. I don't figure
this is a big deal, and can be added in later.

Looking forward to hearing feedback!


Sent through the nmap-dev mailing list
Archived at http://seclists.org/nmap-dev/


Attachment: before.xml

Attachment: after.xml

Sent through the nmap-dev mailing list
Archived at http://seclists.org/nmap-dev/

  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]