Home page logo

nmap-dev logo Nmap Development mailing list archives

Re: [NSE] http-git patches
From: Alex Weber <alexwebr () gmail com>
Date: Tue, 11 Dec 2012 15:50:50 -0600

Thank-you for doing that bisection and fixing the problem.

I made the changes you suggested, and a lot of code was touched, so the
patches are big. While making the changes, I had a code review done and
made some style changes to enhance readability and bring http-git closer
to the style of the other NSE scripts.

The real changes are in http-git_functional.patch , and the
readability/style changes are done in http-git_style.patch .
The style patch needs to be applied after the functional patch.

I thought it might be easier to treat it as a new script because a lot
has changed, so I attached the full text of the script as http-git.nse_new.
This is equivalent to applying the functional patch and then the style
patch to SVNs copy of http-git.

Please do let me know if there's a better way to do big patches like this :)


On 08/12/12 04:49 AM, David Fifield wrote:
On Fri, Dec 07, 2012 at 07:31:59PM -0600, Alex Weber wrote:
http-git seemed to have stop working at some point,

I did a bisection and found that it stopped working at r29570, the
introduction of structured output. What went wrong is that string.gsub
returns two value: the substituted string and the number of
substitutions. This was being interpreted as structured output and
unstructured output, respectively. I fixed this in r30354 by removing
the string.gsub call.

so I took the opportunity to change the output to a table so that we
have some semantic XML with -oX.

Thank you for doing this. We're tring to gradually move existing scripts
to structured output, and this helps.

I'd like you to consider making some changes to the structured output.
Here is what it looks like now:

$ ./nmap -p80 --script http-git javaop.com -d -oX -
<table key="">
  <elem>Potential Git repository found at (found 5 of 6 expected files)</elem>
  <elem>Source might be at https://github.com/skullspace/breachdb</elem>
  <elem key="Interesting files">Contents of '.git/COMMIT_EDITMSG' matched patterns 'passw' (case-insensitive 
  <elem key="Repository description">Unnamed repository; edit this file 'description' to name the...</elem>
  <elem key="Last commit message">Brought the github repo up-to-date with my private subversio...</elem>
  <elem key="Remote">git () github com:skullspace/breachdb.git</elem>

Parts of this could be better structured. For example, the URLs: those
should be isolated in elements of their own, not appear in the middle of
free text. (In general, structured output shouldn't have free text
unless it is something copied from the service.) In normal output you
show "found 5 of 6 interesting files"; in structured output you should
list the names of those files. Strings like "Interesting files" and
"Repository description" are bad key names. Imagine that you are a
programmer processing this data--what is the most convenient form in
which it can appear? The line
      Contents of '.git/COMMIT_EDITMSG' matched patterns 'passw' (case-insensitive match)
should definitely be broken down into subelements:
      <table key="interesting-files">
                      <elem key="filename">.get/COMMIT_EDITMSG</elem>
                      <elem key="pattern">passw</elem>
Strings should not be truncated with "..." in structured output; that's
only for normal screen output. Think of a programmer using this XML: you
want everything to be there.

Generally, this is how you should approach structured output. Have a
function that returns everything interesting the script can report, in a
programmer-oriented form. That means a table with nice key names, and
subtables when you need to represent a list or hierarchy. This table
will be the first return value of your script. Then, make a function
that processes the nice programmer-oriented table and transforms it into
screen output. This is where you do things like omit the names of the 5
expected files, and truncate some strings with "...". The result of the
formatting function will be the second return value of the script. See
http-auth for an example of this: the challenges table is directly
returned from an API function; it's programmer-oriented data. The result
string contains a bunch of free text and labels for screen output.

Please add an @xmloutput section in future patches to show what the
structured output looks like.

David Fifield

Attachment: http-git_functional.patch

Attachment: http-git.nse_new

Attachment: http-git_style.patch

Sent through the dev mailing list
Archived at http://seclists.org/nmap-dev/

  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]