Comparing Scanning Tools

[James.McGovern at thehartford.com (McGovern, James F (HTSC, IT))]

Thanks for the response. One of the things that I have been struggling to understand is not the importance of using 
such a tool as I believe they provide value but more of the fact that these tools may not be financial sustainable.

Many large enterprises nowadays outsource development to third parties. Likewise, the mindset in terms of budgeting 
tends to eschew "per developer seat" tool purchases. Nowadays, it is rare to find an enterprise not using free tools 
such as Eclipse and not paying for IDEs

I have yet to find a large enterprise that has made a significant investment in such tools. I wonder if budgets and the 
tools themselves are really causing more harm than helping in that enterprises will now think about trading off such 
tools vs the expense they cost.

-----Original Message-----
From: leichter_jerrold at emc.com [mailto:leichter_jerrold at emc.com]
Sent: Wednesday, June 07, 2006 4:34 PM
To: McGovern, James F (HTSC, IT)
Cc: sc-l at securecoding.org
Subject: Re: [SC-L] Comparing Scanning Tools


| Date: Mon, 5 Jun 2006 16:50:17 -0400
| From: "McGovern, James F (HTSC, IT)" <James.McGovern at thehartford.com>
| To: sc-l at securecoding.org
| Subject: [SC-L] Comparing Scanning Tools
| 
| The industry analyst take on tools tends to be slightly different than
| software practitioners at times. Curious if anyone has looked at Fortify
and
| has formed any positive / negative / neutral opinions on this tool and
| others...
We evaluated a couple of static code scanning tools internally.  The
following is an extract from an analysis I did.  I've deliberately
omitted comparisons - you want to know about Fortify, not how it
compares to other products (which raises a whole bunch of other
issues), and included the text below.  Standard disclaimers:  This
is not EMC's position, it's my personal take.

Caveats:  This analysis is based on a 3-hour vendor presentation.  The
presenter may have made mistakes, and I certainly don't claim that my
recall of what he said is error-free.  A later discussion with others
familiar with Fortify indicated that the experience we had is typical,
but is not necessarily the right way to evaluate the tool.  Effective
use of Fortify requires building a set of rules appropriate to a
particular environment, method of working, constraints, etc., etc. 
This takes significant time (6 months to a year) and effort, but
it was claimed that once you've put in the effort, Fortify is a
very good security scanner.  I am not in a position to evaluate that
claim myself.

BTW, one thing not called out below is that Fortify can be quite slow.
Our experience in testing was that a Fortify scan took about twice as
long as a C++ compile/link cycle, unless you add "data flow" analysis -
in which case the time is much, much larger.

The brief summary:  In my personal view, Fortify is a worthwhile tool,
but it would not be my first choice.  (Given the opportunity to choose
two tools, it would probably be my second.)  Others involved in the
evaluation reached the opposite conclusion, and rated Fortify first.

                                                        -- Jerry

Fortify


*************************************************************************
This communication, including attachments, is
for the exclusive use of addressee and may contain proprietary,
confidential and/or privileged information.  If you are not the intended
recipient, any use, copying, disclosure, dissemination or distribution is
strictly prohibited.  If you are not the intended recipient, please notify
the sender immediately by return e-mail, delete this communication and
destroy all copies.
*************************************************************************