mailing list archives
Re: Nmap's memory use
From: David Fifield <david () bamsoftware com>
Date: Sat, 19 Dec 2009 15:19:26 -0700
On Thu, Nov 19, 2009 at 06:04:58PM -0700, David Fifield wrote:
We've had some report recently about Nmap using a lot of memory.
"Port memory bloat"
"nmap 5 memory usage"
In the first link above, Pavel Kankovsky observed that it is the Port
class that is using up most of the memory in some scans. My preliminary
tests agree with this. I used the Massif memory profiling too
(http://valgrind.org/docs/manual/ms-manual.html) from the Valgrind suite
Pavel provided a patch that reduces the memory usage of all the Ports in
the default state at the end. I think that is the right idea. I am going
to try to see if there's a way to reduce their memory use even further,
perhaps by changing the programming interface so that the creation of
discrete objects isn't necessary.
I've finished working on memory reduction for now. Apart from the
earlier nmap-os-db parsing memory improvement described in
http://seclists.org/nmap-dev/2009/q4/479, I just merged two more
important improvements from the nmap-mem branch in r16308.
For a bit of context, first look at this benchmark scan, all UDP ports
on four hosts. It uses a maximum of 33.63 MB, almost all of it coming
from addPort at the end of the scan.
The first new improvement was a reduction in size of Port objects. I
used Pavel's idea of dynamically allocating parts of Port that aren't
need for every port, like service and RPC information. The size of a
bare Port object dropped from 92 to 40 bytes on my machine. Now the peak
memory usage is 17.66 MB.
The second improvement is to avoid allocating some Port objects at all.
In many scans, the majority of probes don't receive a response. The big
spike in memory at the end of the previous two graphs is thousands of
Ports being created for the open|filtered ports that didn't respond. Now
the code has a notion of a "default port state" for ports that don't get
a response. All the ports in the default state share a single Port
object. This further reduced memory use to 5.94 MB.
The second improvement won't help, for example, when TCP ports are
closed because of RSTs rather than being filtered, but you will still
get the benefit of smaller Port objects.
Sent through the nmap-dev mailing list
Archived at http://seclists.org/nmap-dev/
- Nmap's memory use David Fifield (Nov 20)
- Re: Nmap's memory use David Fifield (Dec 19)