Re: The CVE (WAS: RE: RE: Ramping up for another review)

[dugsong () monkey org (Dug Song)]

Archive: http://msgs.securepoint.com/ids
FAQ: http://www.ticm.com/kb/faq/idsfaq.html
IDS: http://www-rnks.informatik.tu-cottbus.de/~sobirey/ids.html
UNSUBSCRIBE: email "unsubscribe ids" to majordomo () uow edu au
On Thu, 13 Jul 2000, Greg Shipley wrote:

> Yeah, but the CVE is a start - and it is also my understanding that they 
> are agreeing that the apple is indeed, the apple, and not the orange.

no, this is the problem. they're agreeing that "apple" is a name for some
fruit they should care about, but without any backing definition based on
common taxonomy. vendor A could call a red delicious an apple, while
vendor B could call an orange an apple, and they'd both be CVE-compliant.

i contend that this is worthless for anything more than marketing, and may
actually be harmful in the long run. we've basically given up on science,
yielding instead to market-driven interoperability requirements (or at
least claims to such). mayday, mayday!

while the past vulnerability taxonomy work done at UC Davis, Purdue, and
elsewhere wasn't exactly rocket science, it was definitely a step in the
right direction; CVE, in comparison, seems to be a major step backwards.

> And while I'm on a tangent, I've always found the stuff Max Vision was
> (is?) working on of interest concerning the public postings of basic
> signatures.  You want to talk about hard-core 3rd party evaluation?  
> Look at evaluating the accuracy of vendor signatures.

surely all misuse detection IDS vendors believed their signatures to be
accurate, until simple IDS evasion techniques proved them wrong.

by what measure should we determine accuracy in these matters? the DARPA
IDEVAL project produced simple ROC curves for a given dataset of known
attacks; what about attack permutations and other evasion techniques or
failure modes?

> Another place where the CVE can help - if all IDS vendors become CVE 
> compliant you can make sure to turn on sig X,Y, and Z and know that those 
> are the same across all products (or at least, that they are looking for 
> the same attack) while you test.

not true. the only thing CVE guarantees is that a compliant implementation
has a check for something labelled "apple" (nevermind what kind of fruit
it actually is) - nothing more, nothing less. you could throw a granny
smith at it instead of a red delicious, and it might miss it completely.
or you might throw bananas at it, and have it respond with all kinds of
nonsense about bad apples. so how is this really useful?

if IDS vendors were really concerned with assurance and formal evaluation,
they'd be involved in something like the international Common Criteria:

        http://www.commoncriteria.org/

(not that i think the CC is the end-all be-all of security evaluation,
 but it's a lot more meaningful, imo, than something like CVE or ICSA)

> The CVE could also be another place of easy comparison: see how many
> entries vendor X has covered compared to vendor Y.

as i said before, marketeering. IDS vendors have always played the numbers
game ("we have X more signatures than the competition!"); CVE now lets
them do this with the authority of some imagined standard.

-d.

p.s. sorry if i've stepped on any toes here, i don't mean any offense.
     "business never personal" -- EPMD

http://www.monkey.org/~dugsong/