Speeding up scans that are slow due to wrong rate limiting detection

[Jacek Wielemborek <d33tah () gmail com>]

List,

I had this conversation with David and later with Daniel and I wanted to
spark up more discussion here now. About a month ago, I discovered by
mistake that it is possible for Nmap to slow down to a ridiculous pace
when the rate limiting detection (RLD) algorithm detects enough drops,
which can happen on longer scans [1]. After discussing this with other
developers, here are possible solutions to this problem:

1. Fix the rate limiting detection algorithm. I heard that David tried
that and it turned out to be too dificult due to scan_engine.cc
complexity (and from what I already learned about this file, I can
definitely believe that).

2. As Daniel suggested, some code could be added that opportunistically
tries to decrease the scan delay to counter the rate limiting discovery
mechanism. Of course, we'd need to respect --scan-delay as the minimum
scan delay we can speed up to. I'm not yet sure when exactly should we
speed up (lack of increase in number of probes reaching the host after
scan delay increase due to RLD? some threshold, maybe based on cwnd or
percentage of succesful probes?)

3. My idea is to introduce a magical "speed the scan up" button. During
most scan types, the user can press a button to control output
verbosity. This feels like a good place to control the scan delay from.
My initial idea involved letting the user to increase or decrease the
global scan delay or toggle it using three states - "default", "turned
off for this hostgroup" and "turned off for the whole scan". When RLD
increases scan delay, the user instead of just seeing the following message:

Increasing send delay for 192.168.1.2 from 400 to 800 due to 11 out of
11 dropped probes since last increase.

Would also see:

WARNING: Nmap assumed that dropped probes are the result of rate
limiting. You can speed up scanning by pressing the "r" key if you know
it's not correct.

Daniel advised against changing the scan state at the runtime and David
had doubts about the UI, especially about letting the user know that
this feature is available (I joked that we could highlight the
announcement of this feature in red in the CHANGELOG text file).

David also suggested that we could use some feedback from the Nmap users
that are not the project's developers to avoid solutions that are too
complex. He definitely made a point, because I have to admit I didn't
even know how send delays exactly work before I started digging in
Nmap's code even though I read Fyodor's book. This concept is IMHO very
unintuitive and we could perhaps use some more user warnings.

Do you have thoughts on any of these ideas? Or maybe you could suggest
your own one?

Cheers,
Jacek Wielemborek

  [1]: http://seclists.org/nmap-dev/2014/q3/46

Attachment: signature.asc
Description: OpenPGP digital signature

_______________________________________________
Sent through the dev mailing list
http://nmap.org/mailman/listinfo/dev
Archived at http://seclists.org/nmap-dev/