mailing list archives
some nmap tools
From: MadHat <madhat () unspecific com>
Date: Sat, 6 Dec 2003 21:16:15 -0600
I have the responsibility of monitoring a large number of IPs for
security issues. One of the most important things for me was to know
what was listening where and of course nmap is the only real solution.
The problem was that my boss wanted me to be able to generate a report
of how many new ports were opened in the last 24 hours, how many new
hosts in the past 24 hours, or even how many hosts we have live that
are Internet facing or web servers, etc...
I looked at a lot of the tools out there. Many of them are great, but
did not scale to the size I needed. I ended up writing some simple
scripts in perl to do what I needed to do.
To give an example of what I am doing, I scan 80K+ IPs spread around
the world in about 10 hours from a single, centralized host at this
time. Of these, about 20K+ are accessible from the general Internet.
I do basic scans of the registered ports and do an OS fingerprint. I
have not added Version fingerprinting as of yet.
There are 2 main scripts I use daily, being the nmap-wrapper (which is
really in constant flux trying to find the best configuration) and
nmap-wrapper runs X number of nmap processes in parallel (I currently
use 32). Each process only scans a single host for a few reasons. The
data is taken back into the script and log files are created. Each
class C has it's own log file per day. Not the best solution, but the
easiest and most efficient I could find in a hurry. ( I have looked
into the nmap-sql, but they don't seem to scale well for me. But I am
still looking.) I scan a single host at a time because I want to know
exactly when each host is being scanned (the start and stop time), but
this has been fixed in later versions of nmap and verbose outputs. I
also used a single host this way because, at least in older versions of
nmap, I had issues with timing. It seemed that when trying to do it
all at once, if it hit a block of hosts not responding, it would take
forever to time out (which is where a lot of my tweaking has been) and
not finish in a timely manner. The way nmap wrapper is written if one
process hangs, it keeps going, for as one process ends it launches
another and tries to keep X processes running at all times.
nnmap-diff is run every day after the nmap-wrapper is completed (see
website for example). It does a host-by-host diff from the previous
day (by default it is from 48 hours ago, but you can specify how many
days back the base should be) and will gen a basic report of what
changed (more specifically, by default it only shows what was added and
not was went away). You can also only look for changes on specific
ports, or specific IPs or partial IP matching. It also has the ability
to mail the output to an email address, but I usually just pipe it to
Using these I can detect within 24 hours of when a firewall hole is bad
or a host has some unauthorized app listening on a non-standard port,
etc... in theory anyway and I have detected some good stuff.
There are a couple of other scripts up there as well.
nmap-search, which I have sent to the list before (years ago when I
wrote it), which allows you to search a single nmap -oG output based on
a few different things.
nmap-report, which is similar to the above, but works on the flat file
system generated by nmap-wrapper. I can tell it to give me all the
hosts that have port 80 open, whether or not to include the DNS entry
found by nmap at the time of the run, whether or not to report all the
details found on hosts that match, match on OS, or partial IP (was
going to be subnet, but I have not had a chance to fix it), or just
search on a specific IP. It also allows you to specify how many day
ago to search. The default searches yesterdays DB, but you can tell it
to go back 3 days (-b3) and look for an answer then.
I have been using these for about a year and have had good success thus
far. I am about to start rewriting some of them and recently had the
opportunity to speak to Fyodor about these and some ideas I had and he
recommended I pass them on here as other may want to check them out.
So here it is....
Feedback is welcome as always.
MadHat (at) Unspecific.com
For help using this (nmap-dev) mailing list, send a blank email to
nmap-dev-help () insecure org . List archive: http://seclists.org
- some nmap tools MadHat (Dec 07)