Re: Weird Crash - "WAITING_TO_RUNNING"

[David Fifield <david () bamsoftware com>]

On Tue, Nov 02, 2010 at 04:24:41PM -0600, Nathan wrote:
> On Tue, Nov 2, 2010 at 2:17 PM, Patrick Donnelly <batrick () batbytes com> wrote:
> > Hi Nathan,
> >
> > On Mon, Nov 1, 2010 at 4:12 PM, Nathan <nathan.stocks () gmail com> wrote:
> > > Anyone know why I would get the following crash using the options "-sS
> > > -sV -T4 -p 1-65535" with nmap 5.35DC1?
> > >
> > > "nse_restore: WAITING_TO_RUNNING error!
> > > not enough memory"
> > >
> > > I've got 16GB of RAM on the box (2.6.23 linux kernel), with 10GB
> > > free...so "not enough memory" must not be talking about total system
> > > RAM...
> >
> > It is referring to system memory. I don't know why you got that error
> > though. Do you have ulimit turned on? Are you able to reproduce this
> > error repeatedly and reliably?
>
> To my knowledge, ulimit is not turned on in any fashion:
>
> $ ulimit
> unlimited
>
> I can indeed reproduce this error repeatedly and reliably.  Here's the
> full output of it crashing three times in a row.  Note that it takes
> about 13 minutes until it crashes.
>
> ====> Here's what I did to watch the memory during the third run.
> You'll notice that while a ton of memory was used, there was still
> over 2GB completely free.  There was also a lot of memory in the disk
> cache that could have been used if necessary <====
>
> $ while /bin/true ; do sleep 1 && echo "---------" && date && cat
> /proc/meminfo  | grep MemFree ; done
> [snipped about 13 minutes worth of output right here]
> ---------
> Tue Nov  2 16:04:55 MDT 2010
> MemFree:       2365096 kB              <==== (crash!)
> ---------
> Tue Nov  2 16:04:56 MDT 2010
> MemFree:       5509820 kB

5509820 KB - 2365096 - KB is right at 3 GB, so you could be running out
of per-process address space. Is this on a 32-bit machine?

It looks like there's a loop somewhere that is allocating memory without
bound.

I'd like you to try running each potential script individually to see
which one is responsible. One way is to run Nmap with the -d2 option,
wait five minutes or so, and then press Enter in the terminal. With -d,
Enter causes Nmap to print a stack trace of all running scripts. The
stack trace will look like this:

NSE: Running: 'html-title' (thread: 0x8feea48)
        stack traceback:
                [C]: in function 'receive'
                <More...>

(Another way is to run "nmap -sV -d2" to get a list of scripts, then try
running each of them individually with --script. But I think the "press
Enter" method will be faster.)

David Fifield
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/