mailing list archives
Nmap 3.70 Released: Core Scan Engine Rewrite!
From: Fyodor <fyodor () insecure org>
Date: Tue, 31 Aug 2004 13:17:27 -0700
-----BEGIN PGP SIGNED MESSAGE-----
Nmap's 7th birthday is tomorrow (Sept. 1). Since the proper way to
commemorate it is with a major new release, I am pleased to announce
the immediate availability of Nmap 3.70. This release brings dramatic
changes. The main port scanning engine has been rewritten from
scratch to be faster, scan many hosts in parallel, and be gentler
against target hosts and networks. Service/version detection also now
functions against many hosts in parallel. The UDP system has been
overhauled to work in conjunction with version detection and a new
"open|filtered" state to avoid false open reports against filtered
systems. Nmap now estimates completion time for port/service scans in
verbose mode (-v) when they are expected to take at least a couple
minutes. A "port scan ping" system can dramatically improve scan
times against heavily filtered hosts. There is also a new --exclude
option that allow you to skip given hosts or networks in a broad scan.
For example, a host may be too sensitive or critical to scan at a
given time, or a security admin may not be responsible for certain
subnetworks. Or maybe you want to cease scanning CW.Com IPs because
they keep sending abuse reports to your ISP :). There is a workaround
for the Windows SP2 problems (improved from the workaround in
3.55-SP2), and MAC address reporting now works on Windows. There are
dozens of other changes, which you can read about in the Changelog
Making Nmap faster was one of your top 5 priorities in the last Nmap
user survey, and I hope 3.70 will not disappoint. Timing varies
dramatically based on network/firewall characteristics, but almost all
of the pre-release feedback has been positive. For example, Bill
Peterson, an Information Security Analyst at Alcatel, regularly scans
a million IP addresses over the Internet to keep the company secure.
He reported that with 3.55, "my scans were running for more than two
weeks". He switched to a 3.70 pre-release (incidentally on a somewhat
beefier machine), optimized the option flags a bit, and was soon
finishing the scans in under a day. I've been doing my own testing
against thousands of machines as well. The time taken for the command
"nmap localhost" on my primary development machine improved from 3
seconds to less than three TENTHS of a second. The command "nmap -T4
scanme.insecure.org" (a filtered-by-default machine) over my home aDSL
line improved from 31.8 seconds to 19.7. Results involving multiple
machines or UDP scanning are often far more dramatic than these. The
official motto for this release is: So fast it deserves a CERT
If you do find a case where the UNIX version 3.70 is slower than 3.55,
let me know. I'm afraid that speeds of the Windows version of Nmap
may not have improved as dramatically as on Linux/BSD/Mac OS X. I had
to spend time working around MS SP2 nonsense rather than focusing on
optimizing for that platform.
You might expect that such dramatic changes to the core of Nmap would
load the new release with bugs. There are always some, but I hope you
will be pleasantly surprised by 3.70's stability. The nmap-dev list
has admirably tested many pre-releases over the last few weeks. I
would particularly like to thank Gisle Vanem, Eric of Catastrophe.net,
Andy Lutomirski, Dana Epp, Mark-David McLaughlin, William McVey,
Arturo "Buanzo" Busleiman, Bill Petersen, and Tom Duffy for major
contributions to 3.70.
Here is the full list of significant changes:
o Rewrote core port scanning engine, which is now named ultra_scan().
Improved algorithms make this faster (often dramatically so) in
almost all cases. Not only is it superior against single hosts, but
ultra_scan() can scan many hosts (sometimes hundreds) in parallel.
This offers many efficiency/speed advantages. For example, hosts
often limit the ICMP port unreachable packets used by UDP scans to
1/second. That made those scans extraordinarily slow in previous
versions of Nmap. But if you are scanning 100 hosts at once,
suddenly you can receive 100 responses per second. Spreading the
scan amongst hosts is also gentler toward the target hosts. Nmap
can still scan many ports at the same time, as well. If you find
cases where ultra_scan is slower or less accurate, please send a
report (including exact command-lines, versions used, and output, if
possible) to Fyodor.
o Added --max_hostgroup option which specifies the maximum number of
hosts that Nmap is allowed to scan in parallel.
o Added --min_hostgroup option which specifies the minimum number of
hosts that Nmap should scan in parallel (there are some exceptions
where Nmap will still scan smaller groups -- see man page). Of
course, Nmap will try to choose efficient values even if you don't
specify hostgroup restrictions explicitly.
o Rewrote TCP SYN, ACK, Window, and Connect() scans to use
ultra_scan() framework, rather than the old pos_scan().
o Rewrote FIN, Xmas, NULL, Maimon, UDP, and IP Protocol scans to use
ultra_scan(), rather than the old super_scan().
o Overhauled UDP scan. Ports that don't respond are now classified as
"open|filtered" (open or filtered) rather than "open". The (somewhat
rare) ports that actually respond with a UDP packet to the empty
probe are considered open. If version detection is requested, it
will be performed on open|filtered ports. Any that respond to any of
the UDP probes will have their status changed to open. This avoids a
the false-positive problem where filtered UDP ports appear to be
open, leading to terrified newbies thinking their machine is
infected by back orifice.
o Nmap now estimates completion times for almost all port scan types
(any that use ultra_scan()) as well as service scan (version
detection). These are only shown in verbose mode (-v). On scans
that take more than a minute or two, you will see occasional updates
SYN Stealth Scan Timing: About 30.01% done; ETC: 16:04 (0:01:09 remaining)
New updates are given if the estimates change significantly.
o Added --exclude option, which lets you specify a comma-separated
list of targets (hosts, ranges, netblocks) that should be excluded
from the scan. This is useful to keep from scannig yourself, your
ISP, particularly sensitive hosts, etc. The new --excludefile reads
the list (newline-delimited) from a given file. All the work was
done by Mark-David McLaughlin (mdmcl(a)cisco.com> and William McVey
( wam(a)cisco.com ), who sent me a well-designed and well-tested
o Nmap now has a "port scan ping" system. If it has received at least
one response from any port on the host, but has not received
responses lately (usually due to filtering), Nmap will "ping" that
known-good port occasionally to detect latency, packet drop rate,
o Service/version detection now handles multiple hosts at once for
more efficient and less-intrusive operation.
o Nmap now wishes itself a happy birthday when run on September 1 in
verbose mode! The first public release was on that date in 1997.
o The port randomizer now has a bias toward putting
commonly-accessible ports (80, 22, etc.) near the beginning of the
list. Getting a response early helps Nmap calculate response times and
detect packet loss, so the scan goes faster.
o Host timeout system (--host_timeout) overhauled to support host
parallelization. Hosts times are tracked separately, so a host that
finishes a SYN scan quickly is not penalized for an exceptionally
slow host being scanned at the same time.
o When Nmap has not received any responses from a host, it can now
use certain timing values from other hosts from the same scan
group. This way Nmap doesn't have to use absolute-worst-case
(300bps SLIP link to Uzbekistan) round trip timeouts and such.
o Enabled MAC address reporting when using the Windows version
of Nmap. Thanks to Andy Lutomirski (luto(a)stanford.edu) for
writing and sending the patch.
o Workaround crippled raw sockets on Microsoft Windows XP SP2 scans.
I applied a patch by Andy Lutomirski (luto(a)stanford.edu) which
causes Nmap to default to winpcap sends instead. The winpcap send
functionality was already there for versions of Windows such as NT and
Win98 that never supported Raw Sockets in the first place.
o Changed how Nmap sends Arp requests on Windows to use the iphlpapi
SendARP() function rather than creating it raw and reading the
response from the Windows ARP cache. This works around a
(reasonable) feature of Windows Firewall which ignored such
unsolicited responses. The firewall is turned on by default as of
Windows XP SP2. This change was implemented by Dana Epp
o Fixed some Windows portability issues discovered by Gisle Vanem
o Upgraded libpcap from version 0.7.2 to 0.8.3. This was an attempt
to fix an annoying bug, which I then found was actually in my code
rather than libpcap :).
o Removed Ident scan (-I). It was rarely useful, and the
implementation would have to be rewritten for the new ultra_scan()
system. If there is significant demand, perhaps I'll put it back in
o Documented the --osscan_limit option, which saves time by skipping
OS detection if at least one open and one closed port are not found on
the remote hosts. OS detection is much less reliable against such
hosts anyway, and skipping it can save some time.
o Updated nmapfe.desktop file to provide better NmapFE desktop support
under Fedora Core and other systems. Thanks to Mephisto
(mephisto(a)mephisto.ma.cx) for sending the patch.
o Further nmapfe.desktop changes to better fit the freedesktop
standard. The patch came from Murphy (m3rf(a)swimmingnoodle.com).
o Fixed capitalization (with a perl script) of many over-capitalized
vendor names in nmap-mac-prefixes.
o Ensured that MAC address vendor names are always escaped in XML
output if they contain illegal characters (particularly '&'). Thanks
to Matthieu Verbert (mve(a)zurich.ibm.com) for the report and a patch.
o Changed xmloutputversion in XML output from 1.0 to 1.01 to note that
there was a slight change (which was actually the MAC stuff in 3.55).
Thanks to Lionel CONS (lionel.cons(a)cern.ch) for the suggestion.
o Many Windows portability fix and bug fixes, thanks to patch from
Gisle Vanem (giva(a)bgnett.no). With these changes, he was able to
compile Nmap on Windows using MingW + gcc 3.4 C++ rather than MS
o Removed (addport) tags from XML output. They used to provide open
ports as they were discovered, but don't work now that the port
scanners scan many hosts at once. They did not specify an IP
address. Of course the appropriate (port) tags are still printed
once scanning of a target is complete.
o Configure script now detects GNU/k*BSD systems (whatever those are),
thanks to patch from Robert Millan (rmh () debian org)
o Fixed various crashes and assertion failures related to the new
ultra_scan() system, that were found by Arturo "Buanzo" Busleiman
(buanzo(a)buanzo.com.ar), Eric (catastrophe.net), and Bill Petersen
o Fixed some minor memory leaks relating to ping and list scanning as
well as the Nmap output table. These were found with valgrind (
o Provide limited --packet_trace support for TCP connect() (-sT)
o Fixed compilation on certain Solaris machines thanks to a patch by
Tom Duffy (tduffy(a)sun.com)
o Fixed some warnings that crop up when compiling nbase C files with a
C++ compiler. Thanks to Gisle Vanem (giva(a)bgnett.no) for sending
o Tweaked the License blurb on source files and in the man page. It
clarifies some issues and includes a new GPL exception that
explicitly allows linking with the OpenSSL library. Some people
believe that the GPL and OpenSSL licenses are incompatable without
this special exception.
o Fixed some serious runtime portability issues on *BSD systems.
Thanks to Eric (catastrophe.net) for reporting the problem.
o Changed the argument parser to better detect bogus arguments to the
o Removed a spurious warning message relating to the Windows ARP cache
being empty. Patch by Gisle Vanem (giva(a)bgnett.no).
o Removed some C++-style line comments (//) from nbase, because some C
compilers (particularly on Solaris) barf on those. Problem reported
by Raju Alluri <Raju.Alluri(a)Sun.COM>
As usual, 3.70 is available from
http://www.insecure.org/nmap/nmap_download.html, including Windows
(.zip format) binaries.
For the more paranoid (smart) members of the list, here are the md5
These release notes should be signed with my PGP key, which is
available at http://www.insecure.org/fyodor_gpgkey.txt . The key
fingerprint is: 97 2F 93 AB 9C B0 09 80 D9 51 40 6B B9 BC E1 7E
Enjoy! And please let me know if you find any problems.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org
-----END PGP SIGNATURE-----
For help using this (nmap-hackers) mailing list, send a blank email to
nmap-hackers-help () insecure org . List archive: http://seclists.org
- Nmap 3.70 Released: Core Scan Engine Rewrite! Fyodor (Aug 31)