mailing list archives
Re: SOC idea
From: Jacek Wielemborek <d33tah () gmail com>
Date: Thu, 09 Jan 2014 18:21:04 +0100
See my answer inline below.
09/01/2014 07:40:35 Jason Gerfen <jason.gerfen () gmail com>:
I just posted on the announce list, which led me to
http://nmap.org/soc/#creative and now this message.
They say you miss 100% of the shots you never take so here it goes...
Congratulations, great attitude! I understand that you're interested in Google
Summer of Code. If so, getting one's idea accepted is definitely not that
difficult if you're motivated (I, for one, am a good example :)). You already
did a few things right - got involved early, came up with an idea and
requested comments on the mailing list. So far, way to go, IMHO :)
I have already started a project which utilizes the current nmap
binaries and improves the speed dramatically. The project has only
recently started and with some preliminary benchmarks shaves 20seconds
off simple host discovery scans.
Details of the project can be found at
I took a quick look at your code and if I understand it correctly, it's
currently limited to host discovery. There's not much written yet, but I
already have a few potentially interesting comments:
1. Your Github readme suggests that you might be using Nmap 5.51. If this is
the case, do upgrade to the latest version before benchmarking your code,
otherwise you might be wasting time.
2. Consider using -oX instead of -oG. This way you will get an ouput format
less likely to be changed in the future. As far as I know, -oG is meant for
being read by a human, not parsed by a machine.
3. In addition to using -sn, you might also want to call -n, which disables
rDNS queries. This might speed up the queries to a greater extent.
4. See if you're running Nmap in priviledged mode (one, though insecure, way
to enable it is to run Nmap as root. For better solutions, see
https://secwiki.org/w/Running_nmap_as_an_unprivileged_user). This enables many
speedups, such as - in your discovery case - ARP ping, which is both more
accurate and faster in case of local networks.
It might also be a good idea to do some research on cutting down the Nmap
initialization time - for example by compiling it with a C library lighter
Also, consider the use case you're planning to eventually accomodate. If your
goal is to speed things up, there are things that you can do faster by
spawning multiple Nmap processes than by running a single process. This
usually not due to SMP though - currently, AFAIK, the biggest problem is that
Nmap scanning is split into phases. There's the discovery phase, rDNS phase,
port scanning phase, NSE, traceroute, OS discovery, service discovery etc...
If in any of the hosts lags behind in any of the phases, all remaining ones
have to wait. Currently everybody walks around the problem by scripting stuff
like you do, but we all know that it's more or less a temporary solution.
There are other use cases for your library if you keep developing it - some
things cannot currently be done with pure command line and it's possible that
not all of them will ever be added to Nmap. One of the features missing in the
Nmap core that we might expect to be added to Nmap soon is port specification
that would allow to exclude certain ports. You could probably implement it in
your library more easily than you would do that in the C code (though it's not
the correct solution actually).
Another example is a feature that I once needed, but I would be surprised to
ever find in Nmap core. I needed to perform a lot of OS discoveries on hosts
whose closed/open TCP/UDP ports differed. I wrote a script myself that
performed the probing and I believe that there are many use cases like this
that could be made easier with a cool node.js interface. Keep in mind though
that if you plan to go around problems that way, you have to solve many
problems that Nmap takes care of for you.
One of the problems is timing. When I probed my first few thousands of hosts, I
already got two automated abuse complaints from some trigger-happy IDSes. I
quickly understood that even though I actually sent SYNs to just three ports,
it summed up to numbers big enough for these systems to complain. I then wrote
a script that ordered the targets so that it's less likely for two hosts from
the same network to be probed in a short period of time. That's one of the
things you might have to do for your tool to be useful in larger scans.
My final advice for you is that you really should read Fyodor's book. It's an
amazing guide to Nmap's functionality, with a lot of explanations that will
help you understand what actually happens and when will you need the features.
It's a few hours of reading, but I guarantee you that your knowledge of Nmap
will greatly increase, which you're bound make use of in the future :)
Especially that the default "nmap scanme.nmap.org" turns on quite a lot of
options and many of them are very risky if you want to stay stealthy.
Whoa, quite a lengthy post. I hope I helped and didn't discourage you. Keep up
your work, it might actually prove useful if you think it over!
Description: This is a digitally signed message part.
Sent through the dev mailing list
Archived at http://seclists.org/nmap-dev/
Re: SOC idea A. P. Garcia (Jan 10)
- SOC idea Jason Gerfen (Jan 09)
- Re: SOC idea Jacek Wielemborek (Jan 09)