Re: [NSE] [patch] Big changes to http-enum.nse

[Ron <ron () skullsecurity net>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mon, 18 Oct 2010 13:22:42 -0600 David Fifield <david () bamsoftware com> wrote:
> On Mon, Oct 18, 2010 at 06:25:15AM +0200, Patrik Karlsson wrote:
> > On 17 okt 2010, at 22.55, Ron wrote:
> > > All right, I'm attaching my newest patch (and the
> > > http-fingerprints.lua file separately, in case people just want
> > > to check that out). I think it addresses all the ideas we've
> > > thrown around so far in a pretty nice way. The configuration file
> > > is now a .lua and basically builds a fairly flexible table. There
> > > is a bunch of validation in the script to ensure the person
> > > didn't miss a required field or use an incorrect variable type,
> > > too. 
> > >
> > > I need to go over the fingerprints file and do some cleanup, but
> > > the actual functionality is here now. 
> > >
> > > Comments would be great! 
> >
> > While being a lot more flexible, wouldn't the new format still
> > require four match lines for eg. Outlook Web Access in the
> > following examples?
> >
> > table.insert(fingerprints, { path='/mail/', verb='GET',
> > matches={ {match='*owa*', output='Outlook Web Access'} }})
> > table.insert(fingerprints, { path='/webmail/', verb='GET',
> > matches={ {match='*owa*', output='Outlook Web Access'} }})
> > table.insert(fingerprints, { path='/', verb='GET',
> > matches={ {match='*owa*', output='Outlook Web Access'} }})
> > table.insert(fingerprints, { path='/owa/', verb='GET',
> > matches={ {match='*owa*', output='Outlook Web Access'} }})
> >
> > I was thinking more along the lines:
> >
> > Probe { path="/mail/", verb="GET"  }
> > Probe { path="/webmail/", verb="GET"  }
> > Probe { path="/", verb="GET"  }
> > Probe { path="/owa/", verb="GET"  }
> >
> > match { status="200", body="*owa*", desc="Outlook Web Access" }
> >
> > I might be missing something that makes the choice of splitting the
> > Probe and match like this a very bad idea?
>
> I think that will be too expansive, if you mean that every match will
> be matched against every probe. Like this path for example:
> /archive/flash:home/html/images/Cisco_logo.gif
> It's not a good idea to grep the GIF file for "*owa*".
>
> In nmap-service-probes we have a system where one probe has multiple
> matches. If you want the same match for two different probes, you
> either duplicate the match line, or (better) declare a fallback probe
> (http://nmap.org/book/vscan-technique.html#vscan-cheats-and-fallbacks).
>
> What Ron is describing is a system where multiple paths can share
> multiple matches in common. It's as if each path has a fallback to all
> the others. If it's truly meaningful for a match at different paths to
> mean different things (different versions for example), then the
> matches will have to be duplicated in two different lines.
>
> A benefit of having the database file in Lua is that you can
> procedurally generate a bunch of probes:
> KNOWN_VERSIONS = {"0.99", "1.00", "1.00a", "1.01", "1.999"}
> for _, version in ipairs(KNOWN_VERSIONS)
>       Probe { patch="/app-" .. version .. "/", ... }
> end
>
> As for this format:
>
> table.insert(fingerprints, {
>  path='/phpmyadmin/',
>  verb='GET'
>  matches={
>   {match='PhpMyAdmin (.*)', output='Found PhpMyAdmin version \1'},
>   {output='Found PhpMyAdmin, unknown version'}
>  },
> )
>
> I think that the method ("verb") and path should be combined. Like
> this:
>
> table.insert(fingerprints, {
>  probes={{verb='GET', path='/phpmyadmin/'}},
>  matches={
>   {match='PhpMyAdmin (.*)', output='Found PhpMyAdmin version \1'},
>   {output='Found PhpMyAdmin, unknown version'}
>  },
> )
>
> Multiple paths would be like this:
>
> table.insert(fingerprints, {
>  probes={{verb='GET', path='/phpmyadmin/'},
>          {verb='GET', path='/PhpMyAdmin/'}},
>  matches={
>   {match='PhpMyAdmin (.*)', output='Found PhpMyAdmin version \1'},
>   {output='Found PhpMyAdmin, unknown version'}
>  },
> )
>
> But the default method would be GET, so you could do simply
>
> table.insert(fingerprints, {
>  probes={'/phpmyadmin/', '/PhpMyAdmin/'},
>  matches={
>   {match='PhpMyAdmin (.*)', output='Found PhpMyAdmin version \1'},
>   {output='Found PhpMyAdmin, unknown version'}
>  },
> )
>
> The idea is to flexibly support other information that we want to vary
> along with the path. (Like not forcing all paths to have the same
> method.)
>
> table.insert(fingerprints, {
>  probes={'/phpmyadmin/', '/PhpMyAdmin/',
>          {verb='POST', '/phpmyadmin/whatever',
> postdata={action='list'}}}, matches={
>   {match='PhpMyAdmin (.*)', output='Found PhpMyAdmin version \1'},
>   {output='Found PhpMyAdmin, unknown version'}
>  },
> )
>
> This could also be used to add a rarity to each path à la
> --version-intensity.
Sure, that's a do-able change. I like the idea of expanding a 'probes' table like that, it means we can add extra stuff 
to probes fairly easily, if it becomes necessary (maybe a HTTP header or something). 

I also want to move 'severity' into the match table, so each match can have a different severity rating. Right now, the 
severities are ignored, and maybe we'll never use them, but it's there if we want it. 

I'll work on this tonight and post an updated patch, if I finish it. 

Ron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.16 (GNU/Linux)

iEYEARECAAYFAky8pQcACgkQ2t2zxlt4g/QJxwCgzOj9DgOERA6h8ijJTitwr/ff
QUQAoIGvTZ8I9UhYkvnwfDksij72zCck
=d1Ts
-----END PGP SIGNATURE-----
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/