Home page logo

nmap-dev logo Nmap Development mailing list archives

Re: Timing race cars with a sundial (-sV match performance)
From: Brandon Enright <bmenrigh () ucsd edu>
Date: Sat, 9 May 2009 00:46:57 +0000

Hash: SHA1

On Fri, 8 May 2009 17:27:58 -0700
Fyodor <fyodor () insecure org> wrote:

On Fri, May 08, 2009 at 10:07:03PM +0000, Brandon Enright wrote:
A few weeks ago I embarked on a (mostly) futile attempt to measure
the performance of applying a PCRE match expression in our

Thanks for sending the results, even if they weren't all you hoped.

Actually the results were better than I hoped.  I was under the mistake
impression that a lot of our more inefficient match lines actually
translated to slow matching.  Instead of finding out that we really
need to fix hundreds of matches we learned that we probably only need
to fix ~5 :-)

As for optimizing these matches, for the LPD match we really need to
add an anchor.  I did some checking and all UCSD hosts that match
the service can safely add '^' and still match.

Sounds like a good idea.

The '^.*' usage in the ser2net match is counter-productive.  I don't
have any matching services so I don't know what can be done to
improve on '^.*' besides just removing it.

Removing the "^.*" sounds like the way to go, as it seems completely
pointless to me.

Will do.  It won't actually make the match any faster but it will at
least make study go faster.

The Xylan PizzaSwitch telnetd match is pretty zealous in its use
of .* early in the match.  Telnet services often match the start and
then print a large amount of data (banners, abuse warnings, etc).
'.*' is consuming all of that data on all telnet services and then
backtracking a byte at a time.  We should make the '.*' lazy by
changing it to ".*?". Even better would be to add a few more
matching bytes to match the telnet control bytes before using '.*'
but we may not have enough data to do this.

I'll forward you some Xylan PizzaSwitch signatures in a few minutes in
case it helps to have the extra data.

(I just got them, thanks!)

The Cyrus Murder matches look like a '^' can be added.  The protocol
looks like IMAP and it is safe to anchor the \* in IMAP with '^'.
UCSD doesn't have any Cyrus Murder installs for me to test.  I'd
suggest we add the anchor and then wait for new submissions if it
doesn't match.

Sounds reasonable.

Another user (thanks!) shared with me their server output which looked
something like:

* OK MUPDATE "hostname" "Cyrus Murder" "v2.3.13" "(master)"

This means that although Cyrus Murder is in the spirit of IMAP it
isn't exact and a ^ won't work.

We can still make a failing match /much/ faster though.  We need to
tell PCRE that instead of skipping bytes that couldn't possibly match
(aka .*) the format is "\* something\r\n".

I propose:

m|^(\* [^\r\n]+\r\n)*?\* OK...

PCRE should be able to fail quickly rather than trying to apply the
match to every byte offset in the service.

I'm happy to submit a patch that does all of the above if it sounds

Yeah, please do create such a patch and check it in.

I'll leave the Murder changes above out unless you think it is okay to
put such a regex in the probes file.

We might also think about adding Nmap internal performance
statistics logging to Nmap proper similar to my addition of
log_write(LOG_STATS, ...) in this branch.  I feel like sometimes
- -d3 or more is too much data when all you want to do is measure
performance stats.

Have written up anywhere how this feature works?  We do try to avoid
filling Nmap with too much debugging code, but sometimes it is
definitely worthwhile.

Okay I'll come up with a proposal.


Version: GnuPG v2.0.11 (GNU/Linux)


Sent through the nmap-dev mailing list
Archived at http://SecLists.Org

  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]