Feedback requested - XML XSL transform changes - Changes since last commit on Nov 14th

[Tom Sellers <nmap () fadedcode net>]

Fyodor,

    Here is a summary of the changes I have made to the XSL since the last
SVN commit on Nov 14th. This includes the updates from my last email to the
list on Nov 22.

Please look at the progress so far and let me know if I am on the right track.

I have attached a copy of the updated nmap.xsl file.  There are two other
attachments, both starting with test_20101127.  They are the same file,
but one has been renamed with a .txt extension to improve the odds of it
getting through. HTML files seem to be stripped.

Here is the current status:


   o The host index at the very top of the file is rather unwieldy.  For
      example, a scan of scanme.nmap.org shows "64.13.134.52 /
      scanme.nmap.org / scanme.nmap.org".  In this case, scanme.nmap.org
      is repeated twice.  But even if it wasn't, I think we should limit
      it to one host name like Nmap does in its host scan report header.
      We should probably use the same algorithm Nmap does to decide which
      hostname to use.  In addition, it is hard to mentally parse when the
      IP and host name(s) for a single host are separated with "/", and
      different hosts with "|".  Those look too similar.  I think it would
      be better to use this traditional Nmap-style format:
      "corn02.Stanford.EDU (171.67.216.67)"


The host index has now been changed to the format of "hostname (IP address)".
As far as the hostname portion preference is given to the user supplied
hostname.



   o A red color is used in the host index to denote down hosts, and in
      the port tables to denote closed ports.  Red is often used in reports
      to highlight the most interesting/important information, but closed
      ports and down hosts are among the least interesting entries.  So I
      think they should either use a default color (e.g. plain black) or
      maybe more subtle gray or something could be used.

The color for downed ports has been changed to light gray.  The display
for them has been changed, see comments below.


   o I'm not sure that the traceroute table needs to be green.  Maybe
      just plain white would be fine?


Traceroute data is current green because each of the hops is up.  The
XSL has code to deal with down and unresponsive hops.  The hops that
don't respond are not output to the XML file even though they are
displayed on the console.

Test case:
        sudo nmap -sP --traceroute -oX test.traceroute.xml  www.cnn.com

Question:  Is this as expected / desired?

The traceroute information, when present, has been moved to a click-to-expand
div element that is collapsed by default. When the document is printed this
table is expanded.



   o It might be worth migrating "runstats" and "scan info" sections
      which appear at the bottom into the "scan summary" section which
      appears at the top.  Also, I think the presentation of those
      sections could be improved.  Right now it is just a list of short
      facts like "171 host(s) offline".  You might be able to save
      vertical space by placing the information in a table, or using a
      sentence format like Nmap does ("Nmap done at Sun Nov 14 14:55:55
      2010 -- 259 IP addresses (88 hosts up) scanned in 2029.02
      seconds").

Done, it turns out that the string that Nmap outputs when it finishes
is included in the XML.


   o There is a lot of information in Nmap XML which we don't show in
      normal Nmap results because it is rarely useful.  But since XML
      isn't meant to be read by humans, there is little harm in including
      obscure details like the exact TCP timestamp sequence and IPID
      sequence values.  Just because these are in the XML doesn't mean
      they need to be in the HTML.  For example, does the average user
      really care about the remote OS guess "reference fingerprint line
      number"?  In general, I don't think the HTML needs to prevent more
      by default than normal Nmap output shows you.  But one big advantage
      of the HTML over normal Nmap output is that you could consider
      making a details expander for each host.  Then the user could click
      "+" (or whatever) to see the really obscure stuff.  There could even
      be an "expand all details" button up top.  And for the stuff we do
      want to show, it would be nice to think about whether it can be
      presented in a better way than an enumerated list of short facts
      directly from the XML.

This data has been moved to a new section named "Misc Metrics".  This
is a click-to-expand div element that is collapsed by default. When
the document is printed this table is expanded.


   o Related to the idea above: you could consider omitting the closed
      ports and down hosts by default, unless the user clicked a button to
      add them (there could be a controls/customization section near the
      top, I suppose).  I'm starting to wonder if Nmap should even include
      closed and filtered ports in the host table by default--maybe they
      should just be listed.

The information on down hosts has been moved to a click-to-expand div
element that is collapsed by default. When the document is printed this
table is expanded.


The display for closed and filtered ports has been changed. By default
the information for closed and filtered ports is filtered from the tables.
The column header now has clickable links that will display each.  The
links indicate the counts of each type (closed vs filtered) in the current
table so that the user can see at a glance if there is anything hidden.

There is also a floating box in the lower right hand corner of the display
that contains links that will will toggle showing and hiding of ports in these
states for the entire document.  This floating box contains a link to the
top of the document as well.

When printing the document the printout will reflect the current status
(hidden vs unhidden) of the ports.  The clickable links themselves are
also not output when printing.


Question:  Does the table of ports need to be changed so that closed and
           and filtered ports are always printed as opposed to printing in
           the format that is currently displayed?  My concern here is
           processes that automatically convert documents, for example to
           PDF format.


   o Nmap has a neat way to show how traceroute results differ from that
      of a previously shown host.  If that information is exported to the
      XML, it might be better to show by default than the full table for
      each host (it could have a link to the reference host).  Even better
      might be an option to show either the full table or the
      differences.  Like you could show the differences by default, with
      an expander to show the full table if desired.  If Nmap doesn't
      export this information in the XML, maybe we should.

Nmap does not export this data to the XML.  I haven't made any changes to this.


   o There is a section titled "remote operating system guess" which
      should probably be "guesses" since it usually contains several
      values.  It might be nice if it used text more like Nmap, noting
      that there are no exact matches but here is a list of the closest
      ones.  And I don't think you need to give things like "reference
      fingerprint line number: 9336" for each entry.  Although it looks
      like this section is used if a result is exact too.

'Remote Operating System Guesses' changed to 'Remote Operating System Detection'

The fingerprint reference line has been removed.

The OS output has been modified to be more like the nmap output "OS match (accuracy)"
which is more compact.

The OS Fingerprint block will be displayed if present and the OS is
NOT identified.

If the fingerprint is present in the XML due to -v or -d AND the OS
WAS identified then it will be provided in a collapsed block which
the user can click to view.  These blocks will be EXCLUDED from print
media.


Question: Does the OS fingerprint need to be printed at all? The only
          scenario that I could think of where this would be useful
          would be if the file was 'printed' to digital media such as PDF
          where the user can still copy and paste the data for submission.


   o It would be nice to see OS detection (operating system and/or system
      type) icons for introducing hosts.  But a challenge is that we'd
      prefer not to load them off a 3rd party site like nmap.org because I
      suppose that could be a privacy risk.  It would tell that a user
      from the IP was reading an Nmap scan report and it contains at least
      one of the device type or operating system identified by the logo.
      Isn't there a way to include small images inline by including the
      hex data?

I looked into this (using img src="data: ... ) and could not get
Firefox to display the icon.  Internet Explorer 8 (!) would display
it just fine.  According to Wikipedia most of IE's support for this
tag is limited and started with 7, but it seems to work the best.
Most other browsers appear to support it though.  I will work some
more on this.


   o This would be a big job, but I think it would be a neat feature if
      you could click a control and change the viewing mode from "list of
      hosts with the services accessible on them" to a big list of
      services listing the IP they represent.  They would probably be
      sorted by port number or service name, allowing someone to easily
      (for example) find all the SSH servers.  Then again, maybe at this
      point they should just use Zenmap.

True, Zenmap may be best at this point but I will try to figure out
a good way to do this, though it might be cleaner to do with a different
XSL.


Misc Changes:

Added HTML Doctype of HTML 4.01 Strict, this tidies up parsing and
clarifies rendering behavior.

Various changes to formatting to improve readability of output when
it is printed.

Changed how host index HTML anchors are created in order to deal with
a warning about the name attribute being deprecated.

Refactored host os detection portused and osmatch sections in order to deal
with HTML / XML tag nesting issues.

Scaninfo and runstats code has been removed altogether.

Reformatted most of the document using 2 space indents instead of tabs
to try to be more consistent with the rest of the project.

Please let me know what you think...

Tom

Attachment: nmap.xsl
Description:

Attachment: test_20101127.txt
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/