Re: [RFC][patch] XML structured script output

[Daniel Miller <bonsaiviking () gmail com>]

I've attached 2 xml files, before.xml and after.xml, which I hope I've
sanitized sufficiently from a quick scan of my network (-sC). Note
that these represent the output of scripts that have not been modified
in any way from their current state. Modifying scripts to use the
format recommended by stdnse.format_output will result in more useful
organization of results in the new XML format, and further clarifying
key-value pairs when returning results will make results even more
useful. See the documentation in the patch for stdnse.format_output
for an example:

---Takes a table of output on the commandline and formats it for display to the
-- user. This is basically done by converting an array of nested tables into a
-- string. In addition to numbered array elements, each table can have a 'name'
-- and a 'warning' value. The 'name' will be displayed above the table, and
-- 'warning' will be displayed, with a 'WARNING' tag, if and only if debugging
-- is enabled.
-- 
-- Here's an example of a table:
-- <code>
--   local domains = {}
--   domains['name'] = "DOMAINS"
--   table.insert(domains, {['Domain 1']="example.com"})
--   table.insert(domains, {['Domain 2']="nmap.org"})
-- 
--   local names = {}
--   names['name'] = "NAMES"
--   names['warning'] = "Not all names could be determined!"
--   table.insert(names, "Name 1")
-- 
--   local response = {}
--   table.insert(response, "Apple pie")
--   table.insert(response, domains)
--   table.insert(response, names)
-- 
--   return stdnse.format_output(true, response)
-- </code>
--
-- With debugging enabled, this is the output:
-- <code>
--   Host script results:
--   |  smb-enum-domains:
--   |    Apple pie
--   |    DOMAINS
--   |      Domain 1: example.com
--   |      Domain 2: nmap.org
--   |    NAMES (WARNING: Not all names could be determined!)
--   |_     Name 1
-- </code>
--
-- The same table will also be rendered for XML output as
-- <code>
-- <script id="smb-enum-domains">
--   <elem>Apple pie</elem>
--   <container name="DOMAINS">
--     <elem key="Domain 1">example.com</elem>
--     <elem key="Domain 2">nmap.org</elem>
--   </container>
--   <container name="NAMES" warning="Not all names could be determined!">
--     <elem>Name 1</elem>
--   </container>
-- </script>
-- </code>
--
--@param status A boolean value dictating whether or not the script succeeded.
--              If status is false, and debugging is enabled, 'ERROR'
is prepended
--              to the line. If status is false and debugging is
disabled, no output
--              occurs.
--@param data   The table of output. This table may be modified by the function.
-- @return <code>nil</code>, if <code>data</code> is empty or status is false
--         and debugging is disabled, otherwise a table properly formatted
--         for the Scripting Engine to output.


Dan


On Sun, May 27, 2012 at 10:42 AM, Djalal Harouni <tixxdz () opendz org> wrote:
> Hi Dan,
>
> Thank you for re-sending, can you please send a sample XML file to show
> the difference ?
>
>
> On Mon, May 21, 2012 at 08:57:38PM -0500, Daniel Miller wrote:
> > Congratulations everyone on version 6!
> >
> > For a year or so, I've been looking for feedback on my patch for XML
> > structured output for NSE scripts [1], and as it has been a while, I
> > thought I'd bring it up again. I've fixed a couple bugs since last
> > time, and I've started keeping my local copy of the Nmap source under
> > Git source control, so I can rebase my branch as development continues
> > on the official source. I'm attaching 2 patches: nse-structured.diff
> > has all the code changes to use XML output for scripts, and
> > scripts-structured.diff patches a few scripts to take better advantage
> > of the structured output.
> >
> > As far as impact to script writers, there shouldn't be much, since a
> > table passed to stdnse.format_output() will work as it always has, as
> > will simply returning a string from a script. In this regard, the
> > functionality is largely opt-in. If the patch is accepted, I'll spend
> > time updating existing scripts to structure their output.
> >
> > There is room for improvement: Currently, "horizontal lists" don't
> > work; that is, comma-separated values on one line of normal output
> > should be output as a list of <elem> elements in XML. The base
> > functionality is there, with ScriptOutputNode.set_display(FLOW), but
> > the logic to tie it in as a Lua function is missing. I don't figure
> > this is a big deal, and can be added in later.
> >
> > Looking forward to hearing feedback!
> >
> > Dan
>
>
>
> > _______________________________________________
> > Sent through the nmap-dev mailing list
> > http://cgi.insecure.org/mailman/listinfo/nmap-dev
> > Archived at http://seclists.org/nmap-dev/
>
>
> --
> tixxdz
> http://opendz.org

Attachment: before.xml
Description:

Attachment: after.xml
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/