Re: The Return on Investment of Good Security

[intel96 <intel96 () bellsouth net>]

Hi Adriel,

First, I am not looking to start a flame war with the comments below. 
Second, I am sorry for being so critical of your blog article, but I
just wanted to point out the pitfalls most professional encounter in
trying to determine Security ROI.

***********
Your blog entry "ROI of good security" was a valid attempt in trying to
relay a complex problem to a general audience.  One of the main problems
with your analysis is that it still looks at the problem from the more
traditional approach often used by Business Continuity Planning (BCP)
professionals and CFOs.  Often their approach does not take into
consideration other factors beside the raw numbers (e.g. man hour costs,
downtime costs, etc.).  Looking beyond these numbers is extremely
important when someone is trying to analysis the costs associated with a
breach in security. 

For example, your cited that "the man hours needed to identify every
compromise device" could be use to calculate damages.  If you are like
me, you get paid (e.g. salaried employee) to perform information
security duties, so for the time that I spend looking for these
compromised devices really does not cost my employer any additional wages. 

The same statement is true for these items too:

    * Man hours to reinstall and configure every device
    * Man hours required to check source code for malicious alterations
    * Man hours to monitor network traffic for hits of malicious traffic
      or access
    * Man hours to educate customers

Another issue is the calculate of downtime caused by a security event
(or a offline network).  This method is also used incorrectly, because
those employees impacted will perform other business tasks (e.g. sorting
files, holding meetings, etc.) while the security event is being
mitigated.  Also most businesses are designed to absorbed small business
disruption  issues and I would argue that most security events fall
within this category.   If you do not believe me try to find any major
losses listed in the 10K and 10Q reports for those organizations that
cited millions in losses in the press for the security events "I love
You,"  SQL Slammer, and Blaster. 

You also mentioned the use of "lost of customers" as another method of
measurement in determining Security ROI.  This type of measurement is
VERY complex, because how does a business know why a customer was lost. 
Also how does this measurement take into consideration "new customers,"
"delayed purchases," and "competitor competition."    For example, in
1999 eBay, Amazon, and Buy.com cited that they loss a combined total of 
1.7 billion dollars from a DDoS attacks.  They also cited they lost
customers because of this massive DDoS attack.  It seems that this loss
number is highly inflected, because I do not see a 425 million loss
cited in any of these companies 10K for FY1999 or FY2000.  Also only
e-Bay (FY2000 10-K) cited that a "denial of service" could cause a
business disruption that could impact their business operations.  None
of these businesses cited that a lost of customers occurred from the
DDoS attacks.

You also cited that the cost of conducting a Pentest can be another
factor in determining Security ROI.  I would argue that a business that
is required by some law or regulation to perform such security testing
cannot use the cost of a test to determine Security ROI.  Why? Because
most security testing does not look at the BIG PICTURE.  Security
testing often only takes a snap shot of the security posture of an
organization.   For example,  a misconfigured  web server that allows a
SQL Injection to occur normally points to  more fundamental security and
development (coding) problems within an organization, which are usually
out of scope of a common pentest.  Also what if the security breach was
caused by something that was not tested for?  For example,  what if a
company has a security test performed against their systems and a
security event occurs through a trusted relationship with a third-party
vendor that compromises the first company's systems? 

Your also vaguely mentioned about security testing cost per hour.  While
it is true that any monkey can download or purchase an automated
security testing application, the bigger question is does the monkey
understand what they are using.   I have seen too many security
professional use automated tools and not understand what the tools is
telling them.   For example, I had someone tell me that a Microsoft web
server was running on an IBM main frame many years back.  I asked them
how they determine this and they cited at a tool informed
them.....hum......of course they were incorrect.  They even put it into
to a report, even though the tool was wrong.

TTFN,
Intel96

Adriel T. Desautels wrote:
> Ed,
>     Two very good points. I didn't intend for the article to be
white-paper quality, I only intended for it to help people realize the
value of security.   If you have a way of calculating the cost avoidance
that can result from good security then I'd be happy to write an
additional blog entry on the subject.
> On Jan 6, 2009, at 12:14 PM, Ed Fuller wrote:
>
> Two points to consider:
> 1 - your discussion shows Cost Avoidance not ROI.
> 2 - the attributes for measuring do not show the legal liability, which
> can/will exceed the other expenses.
>
>               Ed Fuller, CISSP, COO/Principal
> Office:    719-488-4500               ed () securityhorizon com
> Facsimile: 719-268-1709       http://www.securityhorizon.com
> Cellular:  719-659-8195                       Copyright 2009
>                   Security Horizon, Inc.
>        "Your Global Information Security Experts"
>
> Adriel T. Desautels wrote:
> > > > Latest blog entry for those who care. This one compares the Return on
> > > > Investment of good security services to the Return on Investment
> of poor
> > > > quality security services.  As usual comments and criticisms are
> welcome
> > > > and appreciated.
> > > >
> > > > Direct link as requested:
> http://snosoft.blogspot.com/2009/01/cost-of-good-security-is-fraction-of.html
> > > >    Adriel T. Desautels
> > > >    ad_lists () netragard com
> > > >        --------------------------------------
> > > >
> > > >    Subscribe to our blog
> > > >        http://snosoft.blogspot.com

>     Adriel T. Desautels
>     ad_lists () netragard com
>         --------------------------------------

>     Subscribe to our blog
>         http://snosoft.blogspot.com