Re: Group crafts rating system for server security

[Chris Brenton <cbrenton () SOVER NET>]

Dan Tobin wrote:
> While I certainly do see the need for a system like this, I have no
> faith that it can happen, especially by folks with so many vested
> interests in the economics of it.

Hummm. I work for Dartmouth college's security institute and we are also
involved with the CIS project. We actually stand the chance of losing
some funding if CIS is successful. We're involved because it's a great
idea that's badly needed.

> For this to happen in the time frame allocated, a gargantuan effort
> would need to be launched,

I agree completely however I've also been amazed at the amount of
resources that have signed up to help out with this project. It really
has turned into a gargantuan effort.

> and then validated with series of controlled experiments.

Yup, it's called "peer review". Hand it out for free and let people beat
on it and tweak as required. Same principal as open source.

> However, as Weld alluded, there is no way to
> control enough variables to make this statistically valid.

Depends on the level of granularity you are looking for. Can you rate
one system a "7" and another an "8", I think so. Can you rate one system
a "8.12" and another "8.13", I think the whole field is far too fluid
for that level of detail.

This also somewhat misses the point of what CIS is all about
(disclaimer: I don't speak for CIS, this is just my read on the whole
thing). CIS appears to be looking at the Internet as one big network and
attempting to perform a risk assessment. IMHO this is no different than
what all of us do with our own little piece of the pie. The idea is to
try and identify the weak points and pull them up to standard with
everything else.

For example, anyone who spends any time in this field quickly identifies
that there are certain types of networks/domains that a lot of attacks
originate from. This tends to be because systems within these realms are
easy pickings for the attacker community. The idea is to baseline the
level of security put into play by organizations that _are not_ being
used as bounce sites and attempt to do what ever is possible to try and
bring the minimal sites up to spec. This may take the form of free
tools, procedures, support, etc.

I personally don't have all the answers but I'm glad to be helping out
an organization that is taking a stab at making a difference for all of
us.

> Further,
> how long is the "single number" going to be valid for?  The security
> posture of a "system", however you want to define it, changes daily.

Again, I agree completely. As with any risk assessment any checks need
to be dated, revision controlled, etc.

*sigh*
Back when I was consulting, the above attitude was always the biggest
problem I had getting security managers to perform regular risk
assessments. They don't want to hear its an on going process. The
attitude was "why bother if I can't just check it once and be done with
it".

BTW, when I need to do risk assessment predictions I look at the
organization's _process_ for remaining secure. If they are diligent at
receiving alert notifications, installing patches, reviewing logs, etc.
you can make a pretty accurate guess that they will remain secure. If a
consultant walked in and set everything up and the systems are now
running on auto-pilot, you can make a pretty accurate guess that at some
point they'll get whacked.

> Put me into that famous category of peoiple actually wanting to add
> "science" back into "Computer Science".

Don't take it personally, but I *really* hate this attitude. Throw some
stones and then try and close with some cute snippet instead of a
solution. If you have a better idea, please sing out. I'm more than
happy to volunteer my time to support any good project that works
towards making us all more secure. That's why I'm working with CIS. It
may or may not be perfect, but its an effort to do the work rather than
throw stones.

> Wow... if it were this easy, I would have finished my PhD long ago
> probably...

<GRIN>...no comments here.

Chris
--
**************************************
cbrenton () sover net

* Mastering Cisco Routers
http://www.amazon.com/exec/obidos/ASIN/078212643X/
* Mastering Network Security
http://www.amazon.com/exec/obidos/ASIN/0782123430/

ISN is hosted by SecurityFocus.com
---
To unsubscribe email LISTSERV () SecurityFocus com with a message body of
"SIGNOFF ISN".