Re: Inconsistency in nmap XML output

[Matt <smp.repicky () gmail com>]

How many people interested in this thread and getting the host down
added to the XML output are using windows to try and figure this stuff
out (keep reading i'm not just windows bashing, windows can do it all
too)?

Seriously, if you're using linux why would you spend all the time
building XML parsers when you can just run 'awk'.  I do nmap scans
regularly and have yet to use the XML output.  Just -oN and -oG for
me, thx.

In my scanning actions i generally run a -sP and pick up known up
hosts.  I'll then run a -sT -sU simple scan (against the 256 hosts -
the -sP hosts) to see if i can get any hosts to respond that wouldn't
respond to ping would show up.  Then i'll  combine that list with the
-sP output and do a full -sT -sU -p 1- against the whole list.  Here's
how to get to that in awk.

make a file of 1 to 254 and just keep that on the side as a template
(include 0 and 255 if you expect to run up against supernetting).  For
those who can't shell script (includes 0 and 255):

#!/bin/bash
i=0
until [ $i -eq 256 ]
do
echo $i >> somefile.lst
let "i = $i + 1"
done

Then wherever you go, all you have to do is run awk on this to add the
remainder of the IP address to the front of the list.

awk '{print "x.x.x." $0}' somefile.lst > somefile2.lst

Then take your -oN output of up computers from the ping sweep which
probably has "Host x.x.x.x appears to be up." and use awk on that to
get the ip address alone.

awk '$1 == "Host" {print $2}' output.on.file > somefile3.lst

Now, get your diff of all the hosts versus those already up.

diff -y somefile3.lst somefile2.lst --suppress-common-lines | awk
'{print $2}'  > somefile4.lst

run nmap (nmap -sT -sU -n -r -v -P0 -oN somefile5.on.lst -oG
somefile5.og.lst -iL somefile4.lst) to get any information outta host
not responding to ping.  If they don't respond to a simple sT and sU
you probably don't want to run a full scan against them.

take the somefile5.on.lst and run awk against it to find out what
hosts actually have something worth probing.

awk '$1 == "Interesting" {print $3} $2 == "open" {print $0}'
somefile5.on.lst > somefile6.lst

Now inject some human interaction and take a second to evaluate what
is there and see if you have a list of just IP addresses or if you
have anything with an open port.  Otherwise you can do more advanced
awking and tailor it to do more,  but i don't usually run up against
more than 10 addresses that actually respond to anything out of a
subnet and i break up my scans to 1 class C at a time for organization
purposes.

So after you've got your list of pinged hosts (somefile3.lst) and your
surprise hosts from your nmap output file (somefile6.lst) combine them
all into somefile7.lst.

Now you're ready for a full nmap scan on anything that came back with
an answer to your original scans.

For those of you on windows who say, yeah that would be great if only
i could do it.... Well, go get awk and diff for windows or use cygwin
which comes with both set up by default.  Oh and a simple way to keep
extra characters from showing up with newline when viewing in windows
vs linux, use .wri instead of .lst and when you open the file in
windows it will open in wordpad and not show the extra CR.

And if for some reason you still need to know what hosts are down,
then do another diff on somefile7.lst and somefile2.lst.

diff -y somefile7.lst somefile2.lst --suppress-common-lines | awk
'{print $2}' > somefile8.lst  (for those who still need their hands
held)

So who needs XML?  I don't consider nmap to be an end all be all to
build a report from; it's just a middle step.  So I'm interested in
the output not making a report.  And i can search through the -oN much
quicker with awk than going through the XML any other way.  Maybe i've
got a very limited view of nmap, but it has served me well for what
i've been using it for.

I agree with Joshua, how hard was all of what's above?  Not much.  And
Thx to Fyodor for the great program.


--

On Wed, 10 Nov 2004 10:11:55 -0500, Joshua T. Corbin
<jcorbin () linuxmail org> wrote:
> On Wed November 10 2004 02:57, Dual Mobius wrote "Re: Inconsistency in nmap
> XML output":
> > I would also like to see the option of including down hosts in the XML
> > output when -v is used.
> That would be a pain in my opinion; perhaps it should be a separate command
> line option *if* it has to be done at all. I for one, don't see how it makes
> the XML inconsistent in any way.
>
> > (I've previously had to do several nasty hacks in analysis programs to
> > figure out what was reported down vs. not-scanned by taking the address
> > ranges in the <nmaprun> tag and subtracting out the "up" results in the rest
> > of the file).
> How is that a dirty hack, sounds like rather simple math to me. If you need
> the not scanned, you need to look at <nmaprun> anyhow, unless you are
> suggesting that Fyodor implement a --noisy-xml-output option that makes it
> report not-scanned hosts as well, just so your interpreted-language program
> can be a little more lazy.
>
> > I agree with Fyodor's suggestion that if this is added, it should only be
> > when the -v option is used.
> If it's added, it should be a separate option, or maybe turned on be -vvv not
> just -v. And that's a big if, I for one think it would be silly to waste time
> adding clutter to the XML output.
>
> --
> Regards,
> Joshua T. Corbin <jcorbin () linuxmail org>
> 570.274.7693
>
>
>
> ---------------------------------------------------------------------
> For help using this (nmap-dev) mailing list, send a blank email to
> nmap-dev-help () insecure org . List archive: http://seclists.org

---------------------------------------------------------------------
For help using this (nmap-dev) mailing list, send a blank email to 
nmap-dev-help () insecure org . List archive: http://seclists.org