WebApp Sec: RE: OS XSS and SQL scanner

["Burke, Charles" <Charles_Burke () HomeDepot com>]

Dean,
This sounds like a good debate for an upcoming chapter meeting!

I have lead SDLC efforts for commercial software and retail software
companies.
You must perform code reviews and TMs when building commercial software
(apples)
But if my team is developing a specialty eBusiness application that must
be online ASAP to capture
Millions in sales (oranges)? I must trust my pen tester. 

I know that you get the most benefit from code reviews, TM, and process
change!
Question is when and where do you invest the $?

-----Original Message-----
From: Dean H. Saxe [mailto:dean () fullfrontalnerdity com] 
Sent: Wednesday, August 02, 2006 11:14 AM
To: Burke, Charles
Cc: webappsec () securityfocus com; arian.evans () anachronic com;
owasp-atlanta () lists owasp org
Subject: Re: OS XSS and SQL scanner


Charles,

I think we are comparing apples to apples.  I agree, poor quality  
people result in poor quality work no matter what the methodology.

I also understand the "typical security shop", we've spoken about  
this many times.  And I understand your specific needs, too.  I'd  
like to see a cost benefit analysis of all three methods along with  
appropriate changes to the SDLC so we can settle the debate once and  
for all.

-dhs

Dean H. Saxe, CISSP, CEH
dean () fullfrontalnerdity com
"Free speech exercised both individually and through a free press, is  
a necessity in any country where people are themselves free."
     -- Theodore Roosevelt, 1918


On Aug 2, 2006, at 9:12 AM, Burke, Charles wrote:

> Guys,
> Lets compare apples to apples!
>
> Both Web App Scanners and Code Review/Threat Modeling are ineffective 
> when performed by engineers with little knowledge of Web Application 
> Security AND the Software Development Methods/ Language.
>
> If you are working in a typical Security shop and you have an enormous
> workload then you have no choice!  Find the Easy Button, quickly
> review
> results,
> Remediate or Rubber Stamp! THEN move on to the next 20+ assessments
> waiting in your queue.
>
> If you have a project/application with the potential for significant 
> risk/damage then you make the call to Threat Model/Security Code
> Review!
> In some instances you may team with Architecture to sit in on  
> functional
> code reviews to inject security knowledge and assessment.
>
> FWIW
>
> Charles Burke
> Atlanta OWASP Chair
>
> -----Original Message-----
> From: Dean H. Saxe [mailto:dean () fullfrontalnerdity com]
> Sent: Wednesday, August 02, 2006 3:15 AM
> To: webappsec () securityfocus com Security
> Cc: arian.evans () anachronic com
> Subject: Re: OS XSS and SQL scanner
>
>
> Here, here, Arian.
>
> Let's see the web app scanner folks go up against a manual pen test 
> and code review/threat model on a series of apps.  One caveat: the 
> results must be open for review, which means publishing the results in
> an open forum for all to see.
>
> FWIW, I'm a former customer of SPIDynamics.  I have experience web app
> scanners in an enterprise environment along with pen testing and code 
> reviews.  I have a good idea how things will shake out:  Web app 
> scanners are inexpensive to run but don't find significant numbers of 
> vulnerabilities.  Pen tests are a decent measure of security at a 
> reasonable cost when performed my talented testers.  Code review && 
> threat model finds the most vulnerabilities at the highest cost when 
> performed by talented reviewers.
>
> Will any web app scanner companies actually subject their scanners to 
> such a bake off?  If not, how can we trust the marketing material? Was
> Gary McGraw right in calling these tools "badnessometers"?
>
> I'm at BlackHat all week.  Email me and we'll get together and chat. 
> I'll be attending the WASC gathering at Shadow Bar tomorrow night.  I 
> hope to see some of you there.
>
> -dhs
>
> Dean H. Saxe, CISSP,  CEH
> dean () fullfrontalnerdity com
> "[T]he people can always be brought to the bidding of the leaders. 
> This is easy. All you have to do is to tell them they are being 
> attacked, and denounce the pacifists for lack of patriotism and 
> exposing the country to danger. It works the same in every country."
>      --Hermann Goering, Hitler's Reich-Marshall at the Nuremberg
> Trials
>
>
> On Aug 1, 2006, at 2:35 PM, Arian J. Evans wrote:
>
> > > -----Original Message-----
> > > From: Mandeep Khera [mailto:mandeep () cenzic com]
> > >
> > > I am sorry to hear that you perceive some problems with our product.
> > > We take pride in being the most accurate product with least
> > > amount of
> > > false positives in the industry. This has been proven in many 
> > > bake-offs by customers and independent journalists.
> > Hate to take this a little off topic, but do you have any facts that 
> > can support or back up these claims? Any data produced by anyone 
> > competent that speaks to your "false positives" and also your "false 
> > negatives"?
> >
> > I have failed to read a review yet to date that contains useful 
> > information. So far what I've read varies from useless data organized
> > around features like "reflective buttons" (e.g.-the Acunetix review 
> > posted to this list written by some woman who writes windows software
> > articles) to the other extreme of uninformed opinion and inability to
> > keep features between the products straight (secure enterprise 
> > computing review). This includes infosec magazine and online reviews,
> > bake-offs, and Gartner-style evals. Every one I have read so far is 
> > garbage.
> >
> > Not one covers actual tests run & and the how & why around them.
> >
> > This situation is no doubt due to the utter lack of skill and 
> > understanding of the subject on the part of the authors.
> >
> > However, I think all on this list would welcome information of a 
> > high-quality nature regarding scanner quality, if you have anything 
> > like that to point us at.
> >
> > -ae
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > -
> > ---
> > Sponsored by: Watchfire
> >
> > Do you test web applications for XSS, SQL Injections, Buffer 
> > Overflows, Logical issues and other web application security threats?
> > Why not automate this work with Watchfire's AppScan, the world's 
> > leading automated web application scanner. Download AppScan today!
> >
> > https://www.watchfire.com/securearea/appscancamp.aspx?
> > id=701300000008BP9
> > ---------------------------------------------------------------------
> > -
> > ----
>
> ----------------------------------------------------------------------
> --
> -
> Sponsored by: Watchfire
>
> Do you test web applications for XSS, SQL Injections, Buffer
> Overflows,
> Logical issues and other web application security threats? Why not
> automate this work with Watchfire's AppScan, the world's leading
> automated web application scanner. Download AppScan today!
>
> https://www.watchfire.com/securearea/appscancamp.aspx?
> id=701300000008BP9
> ----------------------------------------------------------------------
> --
> --

-------------------------------------------------------------------------
Sponsored by: Watchfire

Do you test web applications for XSS, SQL Injections, Buffer Overflows,
Logical issues and other web application security threats? Why not
automate this work with Watchfire's AppScan, the world's leading
automated web application scanner. Download AppScan today!

https://www.watchfire.com/securearea/appscancamp.aspx?id=701300000008BP9
--------------------------------------------------------------------------