Re: [NSE] Vulnerability Scan based on osvdb

[David Fifield <david () bamsoftware com>]

On Fri, May 21, 2010 at 08:55:25AM +0200, Marc Ruef wrote:
> Your example with Apache proves the reasonability. But when nmap  
> determines "Microsoft IIS httpd 7.0" I've got a new problem.

The "7.0" part of that will actually be separate; port.version.version
instead of port.version.product. But the "Microsoft" problem remains.

> Because the  vendor name "Microsoft" is prefixed, I would also have to
> cut the  preceding strings. This increases the amount of iterations I
> would have  to do. Because in the case of IIS I would have to do the
> following  transformation:
>
> 1 Microsoft IIS httpd 7.0 [vendor + prod + (hum info) + ver]=> no match
> 2 Microsoft IIS httpd     [vendor + product + (human info)] => no match
> 3 Microsoft IIS           [vendor + product]                => no match
> 4 Microsoft               [vendor]            => match (false-positive)
> -
> 5 IIS httpd 7.0           [product + (human info) + version]=> no match
> 6 IIS httpd               [product + (human info)]          => no match
> 7 IIS                     [product]                       => best match
>
> As you can see, this algorithm makes sense so far. But there might be  
> two cases in which we will get wrong results:
>
> 1. If a vendor has two words for vendor names.
>    => false-negative
>       iteration 5sqq. does not help anymore
>       trivia: object_vendors counts 2494 with two or more words
>
> 2. If a product has no vendor but two words in product name and the  
> second string is a common word (e.g. "webserver" or "httpd").
>    => false-positive
>       in iteration 7 (or even 6)
>       example: "Apache httpd" => "httpd"
>
> A possible solution would be to do a replacement of vendor names before  
> the proposed iteration. However, there will be more inconsistencies  
> which will prevent the elimination of all false-positives and  
> false-negatives.

Maybe you can have a canonicalization table of common products that
converts them to your preferred name.

local CANON_TABLE = {
        { "%siis%s", "IIS" },
        { "%sapache tomcat%s", "Tomcat" },
        { "%sapache%s", "Apache" },
}
function canonicalize(name)
        -- Facilitate word boundary detection with %s.
        local match_name = " " .. string.lower(name) .. " "
        for _, pair in ipairs(CANON_TABLE) do
                local pat, canon = unpack(pair)
                if string.find(match_name, pat) then
                        return canon
                end
        end
        return name
end
local NAMES = { "Microsoft IIS httpd", "Apache httpd", "Apache Tomcat httpd", "thttpd" }
for _, name in ipairs(NAMES) do
        print(string.format("\"%s\" -> \"%s\"", name, canonicalize(name)))
end

The output of this program is

"Microsoft IIS httpd" -> "IIS"
"Apache httpd" -> "Apache"
"Apache Tomcat httpd" -> "Tomcat"
"thttpd" -> "thttpd"

> > Software names tend to be pretty distinct. It should be possible to get
> > good confidence with just pattern matching and maybe some
> > canonicalization. We do strive for consistency in nmap-service-probes,
> > but it's a big database and has had several maintainers, which I'm sure
> > is true of OSVBD as well.
>
> I am going to do some more experiments which shall reveal the best  
> approach. On a long-term view the support of CPE still seems to be the  
> best decision.

I have to say that personally, I don't see the use of CPE happening. It
would be nice, but not nice enough to justify what I expect will be
enormous maintenance costs. Also we never did research to see if there's
a similar system that would suit us better.

David Fifield
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/