mailing list archives
Re: [Yokoso-devel] Updates to http-enum.nse
From: Kevin Johnson <kevin () inguardians com>
Date: Fri, 21 Aug 2009 12:38:27 -0400
On Aug 21, 2009, at 10:09 AM, Ron wrote:
I am one of the lead developers for Yokoso! so I thought I should
respond to some of these points here.
I'm cross-posting this to nmap-dev and yokoso-devel, since it sort of
affects both the products. I don't really know what etiquette is
regarding cross-posting, so hopefully that isn't a no-no. :)
I don't mind it at all. And am also cross-posting. ;-)
On 08/21/2009 02:49 AM, Fyodor wrote:
On Thu, Aug 20, 2009 at 11:57:43AM -0500, Ron wrote:
I agree that the script should only show 200 by default, but give an
option to show others. That makes sense. You're going to miss hidden
folders, by by default Apache hides certain things so that kinda makes
For the average person, just giving 200 makes sense, and for the
non-average who understand the extra errors, they can enable a
This idea makes sense to me.
The format is a bit odd, and I'd talked to Kevin a bit about
to have the ability to AND things together (so if a host contains
The format of the fingerprint file is a bit questionable. Comments
lines starting with '#' are parsed and then printed in the script
output when paths given later in the file are discovered. I realize
you didn't invent this format, but it is so simple that it could
easily be improved. For example, each path could include the
description on the same line. Or there could be a keyword
the description on one line, followed by the paths on their own lines
as they are now. Then we could use comments for notes which we don't
want parsed and printed by http-enum. In deciding on the format, it
may be worth thinking about how it could be extended if we want to
include more information later (just as an idea off the top of my
head, we might want to later indicate what status code we look for to
address issues such as http-enum reporting that scanme.nmap.org has
"TeraStation PRO RAID 0/1/5 Network Attached Storage" just because
/cgi-bin/image/shikaku2.png shows forbidden).
4 files that have common names, but are unique as a combo, it'll check
for all of them). At the same time, maybe we can consider other format
changes. Maybe I'll think about it and propose something, see where
Regarding including the status code, that is sort of an issue with the
different purposes of Yokoso vs Nmap. Yokoso doesn't care what the
status code is, but rather a binary: the user has been to the page, or
the user hasn't. Nmap, on the other hand, cares of the page exists.
sure there must be a way to get the best of both worlds, of course --
maybe optional fields or something?
As the guy who created the "format" of the fingerprint file, I think
its important to realize that it was just a file. I like the idea of
something similar to the nikto DB where we include the status code we
expect and a description on the same line. Comments could then be
used for notes to end users and ignored by scripts.
Another serious issue involves inclusion of the Yokoso DB. You say:
That last point is the interesting one, to me -- we use the same
format as the Yokoso project (by Kevin Johnson and others, from
BTW its InGuardians. Not relevant to this discussion but important to
This lets us leverage their fingerprints as well (and
they've given me permission to include a copy of their fingerprints
That was nice of them, but it is important to get more clarification
and more explicit permission whenever we include 3rd party code or
data into Nmap. I hate dealing with copyright stuff as much as the
next guy, but we really need to be very careful about this sort of
thing. When they say we can include the DB with Nmap, what does that
really mean. Remember that Nmap is open source, so people can
incorporate parts into other projects or fork Nmap under a different
name. A strict reading of "you may include this file within Nmap"
would not allow such things, which would mean that part of Nmap is
open source. Also, a strict reading might mean that we can only
include the file and not modify it (create derivative works). In
general, we can put third party code/data in with Nmap if it is given
to us under one of the following licenses (either via special
permission or because the code is already under such a license):
o Public domain -- that means people can do whatever they want with
o BSD-style (includes MIT license, Lua license, etc.) - preferably
2-clause variant. If it has the advertising clause, we need to
mention it explicitly in the man page
potentially other places.
o Nmap license - if they're OK with us distributing it under the
of the Nmap license (http://nmap.org/data/COPYING), that is OK too.
So if they let us use the data under one of these licenses, inclusion
with Nmap is OK. In any case, the license permission granted needs
be included (described) at the top of the file. We only need license
rights to the list of URL paths and descriptions, and not the rest of
Note that even when a data file isn't licensed appropriately for
distribution with Nmap, we can generally point to it in the
documentation (e.g. if it is a URL somewhere) so users can download
and use it.
I'll leave that question for the Yokoso guys to answer.
We currently use the GPL for Yokoso! but have no problems with the
NMAP license and would love to see it included with that in NMAP.
You're probably right about combining the databases. My concern is
it'll make it more difficult to update when Yokoso is updated --
One thought I had -- http-enum.nse and Yokoso sort of have different
points. http-enum.nse is designed for finding common locations, like
/icons, /scripts, /test, etc, and Yokoso is designed for
common web apps. So, for that reason, it might make sense to put
it in a
different script that the user can run separately. Or maybe not. I'm
happy with going either way.
I'm not sure, but my gut reaction is that with a good file format
(which doesn't have to be too complex), we could probably combine the
Yokoso DB with the existing enum DB. There is also a DB by HD Moore
(a NASL script he wrote) which I hope to request permission for us to
If we end up with more URLs than we want to scan by default, we could
look at either splitting it up into multiple scripts or introducing
NSE arguments to select categories of paths to try.
promises to be an active project, and if they're updating the
fingerprints and we've combined the fingerprints, we might run into
I think the best bet is to have multiple fingerprint files of the same
format. Yokoso, defaults, extended, etc. Then the script can load
whatever it sees fit.
I'd definitely like to use HD's database if it's for the same purpose!
I can take responsibility to keep the yokoso fingerprints updated in
the nmap file if you would like!
Thanks for including us, and if there are more questions feel free to
Senior Security Analyst
Description: This is a digitally signed message part
Sent through the nmap-dev mailing list
Archived at http://SecLists.Org
Re: Updates to http-enum.nse Ron (Aug 22)