Home page logo
/

pen-test logo Penetration Testing mailing list archives

Re: Pentesting tool - Commercial
From: Ivan Arce <ivan.arce () coresecurity com>
Date: Thu, 28 Feb 2008 21:26:09 -0200

Hello Andre & pen-testers

Since Core was been mentioned as having lied on this list I felt compelled to reply and ask you to better qualify that statement so I can verify it and address it properly if need be.

I'd also like add my comments regarding your assessment of CORE IMPACT capabilities with the *obvious caveat* that I am Core's CTO. But before that, I'd like to ask you to clarify how did you come to your conclusions and if you were or are a licensed user of a current and up-to-date version of CORE IMPACT because I suspect you may be providing opinions that are based on a partial or limited view of our product. Please feel free to contact me directly or through any of Core's Customer Support channels so we can follow up on any particular feedback (or complain) you may want to provide.

We really appreciate constructive criticism and value the help of the security community to improve our product, then again, many may choose not to help us and that is totally fair too.

Going back to the original comments about CORE IMPACT and the 'count of exploits' I'd like point out just that throwing numbers without qualifying the measurement criteria and the relevance of the methodology is not a very serious assessment of a product's capabilities, its suitability for a given use or the value it may provide to a security professional.

Nonetheless, the last time I checked (a few minutes ago as of today) a fully-functional legitimately activated and up-to-date CORE IMPACT 7.5 distribution has 665 exploits targeting 4414 unique target combinations (OS, SP, service, application version) distributed across 466 modules and that's not counting the components that detect and exploit SQL injection and Remote File Inclusion vulnerabilities in custom/proprietary web applications.

Note that, all of our exploits are fully-tested to meet quality standards before they ship to all our active customers and thereafter continue to be supported through the entire life cycle of the product (all of which is included in the product's purchase and licensing terms). We do not ship 0day exploits (that is, exploits that target vulnerabilities for which information is not publicly available without access or confidentiality restrictions), we do not ship different sets of exploits for subsets of users in our customer base and we do not ship or support exploits (or other modules) that are not developed by our own team, so it is not so easy to a provide fair comparison just with "count numbers".

Exploit quality assurance seems to be an often overlooked topic, perhaps because we are the only organization (that I know of) that does full system tests running all the supported exploits against all the supported platform combinations with all the supported connection methods *daily* in order to have constant visibility of the quality of the "exploits" component of our product (there are many "non-exploit" modules that provide valuable penetration testing capabilities as well). This requires a level of investment in processes and infrastructure that is not insignificant and perhaps only available to a commercial organization that is fully committed to ensure a certain degree of product quality.

Lastly, as I said above, we maintain all the modules shipped with the product and provide regular product updates on a weekly basis. The number of module updates (which include new exploit modules as well as improvement and bug fixes to existing ones and other types of modules) has been at an average of around 22.5 per month in the past 6 months.

But of course... all of this information is provided directly to all our users in the product's dashboard (the first screen that you see when you run CORE IMPACT) so I am merely transcribing it here without much analysis so far.

In any case, I don't know what is the "right" criteria to use for counting of exploits or what's the criteria to determine their relevance, although I've talked about this and presented several ideas at many public venues in the past years and I'd welcome further discussion on that topic.

However, I do think that throwing numbers without a consistent and sound rationale for a comparison is usually miss-leading. In fact I believe this because that had been the case during the early 1990's when "number of vulnerability checks" was the common currency for comparisons of vulnerability scanning products. At the time I was part of the development team of one of the three emerging leaders in the vulnerability scanning software market and I clearly remember that every vendor and all otherwise biased parties had their own (self-serving) methodology and product marketing strategies to count "vuln. checks". In the end those comparisons did not prove very useful to anybody and today there aren't many organizations that use number of vulnerability checks as substantially relevant information to determine if a given scanner will help them fulfill their requirements and provide them the value expected from it.

So that moves the evaluation criteria discussion to a discussion about whether a given product may or may not provide its customer with enough value to justify its purchase and subsequent use and whether it appropriately fits what its expected from it.

I can't speak for other tools or even for CORE IMPACT because I am not a paying customer :) but I can certainly say that at Core we view our product as much more than a bunch of features and a bag of exploits and we make a very serious effort to understand our customer's needs and to combine all our capabilities to build software that gives back to our users the money they paid (and hopefully more!) in the form of valuable and usable tools to improve their security or the security of their customers.

We may or may not succeed in that endeavor but we firmly believe -across our entire organization- that to be the promise that commercial penetration testing software vendors must deliver on.

-ivan


Andre Gironda wrote:
On Wed, Feb 27, 2008 at 1:38 PM, Trygve Aasheim <trygve () pogostick net> wrote:
 This doesn't mean I don't like or use Metasploit, Canvas or any
 other...I just want to point out that the quality of a product is not
 based a number, and Core Impact has proven its quality many times, and
 in many ways.

The numbers show that Core Impact is superior to Canvas and Metasploit.

Unfortunately, it also shows that Impact is missing quite a lot.  The
point I was trying to make is that you can't use only one exploitation
engine.

However, I also fail to see the point of using an exploitation engine
except in the case of testing IPS/IDS or similar.  In this case,
anyone would clearly be better off using BreakingPoint Systems
BPS-1000.

Using exploits on production or IT networks is unethical.  This isn't
the wild west.  You're overpaying by about $19K-$26K for what you need
when you go with Core Impact.  I don't know about ya'll, but the idea
of propagating a pseudo-worm through a corporate network seems about
as good of an idea as asking the power company to shut off electricity
to a hospital for "just a minute, to see what will happen".

Instead of RPT, I suggest asset management combined with regular,
good-old fashioned vulnerability scanning.  Most of the "experts" I
know don't even understand the difference between a vulnerability and
an exploit.  More of those people don't even understand how unreliable
exploits usually are (let alone scanning errors in vulnerability-only
scanners).

Core already lied once on this list about how many modules vs.
exploits vs. CVE's they support.  They could make anything up.  The
money numbers do not lie.  Compare to Rapid7, Tenable, Lumension, or
McAfee for yourself.

If you have to raise awareness by running live exploits, try
Metasploit.  It's free.  Management still not convinced?  Already
covered all the Metasploit exploits?  Try Canvas, it's cheap.
Management still not convinced?  Already covered the Canvas exploits,
too?  Add an exploitation pack or two.  Start writing your own
exploits.  Management still not convinced?  Already covered all of the
Canvas exploitation packs and started writing your own in-house
exploits specific to your architecture?  Maybe Core Impact will help;
call them for a demo.

I have no idea why people are so quick to jump to Core Impact first.
You can't just throw money at these types of problems.  Security is a
very careful and gradual process.

Cheers,
Andre

------------------------------------------------------------------------
This list is sponsored by: Cenzic

Need to secure your web apps NOW?
Cenzic finds more, "real" vulnerabilities fast.
Click to try it, buy it or download a solution FREE today!

http://www.cenzic.com/downloads
------------------------------------------------------------------------


--
"Buy the ticket, take the ride" -HST

Ivan Arce
CTO

CORE SECURITY TECHNOLOGIES
http://www.coresecurity.com

PGP Fingerprint: C7A8 ED85 8D7B 9ADC 6836  B25D 207B E78E 2AD1 F65A


------------------------------------------------------------------------
This list is sponsored by: Cenzic

Need to secure your web apps NOW?
Cenzic finds more, "real" vulnerabilities fast.
Click to try it, buy it or download a solution FREE today!

http://www.cenzic.com/downloads
------------------------------------------------------------------------


  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]