Penetration Testing: Verify Your Security Provider -- The truth behind manual testing.

["Adriel T. Desautels" <ad_lists () netragard com>]

A recent blog entry that we thought "some" of you pen-testers might  
find interesting.  Feel free to leave comments on the blog:
Direct URL: http://snosoft.blogspot.com/2009/07/truth-behind-manual-testing.html


Verify Your Security Provider -- The truth behind manual testing.
Something that I’ve been preaching for a while is that automated  
vulnerability scanners do not produce quality results and as such  
shouldn’t be relied on for penetration tests or vulnerability  
assessments. I’ve been telling people that they should look for a  
security company that offers manual testing, not just automated scans.  
The price points for quality work will be significantly higher, but in  
the end the value is much greater. After all the cost in damages of a  
single successful compromise is far greater than the cost of the best  
possible security services.
I’ve noticed that there are a bunch of vendors who claim to be  
performing manual testing. But when I dig into their methodologies  
their manual testing isn’t real manual testing at all, its just  
vetting of automated scanner results or testing based on the results.  
In other words they test on what the automated scanner reports and  
don’t do any real manual discovery. I’m not saying that tools like  
nessus (an automated scanner) don’t have their place, I’m just saying  
that they aren’t going to protect you from the bad guys. If you want  
to be protected from the threat, you need to be tested at a level that  
is a few notches higher than the threat that you are likely to face in  
the real world.
This is akin to how the Department of Defense tests the armor on its  
tanks, and I’ve probably mentioned this before somewhere on the blog.  
But, we don’t test our tanks against fire from bb guns and .22 caliber  
pistols. If we did that they wouldn’t be very effective in war.We test  
the tanks against a threat that is a few levels higher in intensity  
than what they are likely to face in the real world. As a result, the  
tank can withstand most threats and is a very effective weapon. Doing  
anything less isn’t going to protect you when the threat tries to  
align with your risks; you’ll end up being an expensive casualty of war.
So why do some security companies test at this lesser level? Its  
simple really, they are in the business of making money and care more  
about that then they do about actually protecting their customer’s  
infrastructure. Additionally, there is a market for that sort of low  
quality testing. There are some businesses that don’t actually care  
about their security posture; they just care about passing the test so  
that they can put a check in their compliancy box. Then there are  
other businesses that unknowingly get taken advantage by of vendors  
because they don’t know the difference between high quality and low  
quality services.
So what is the difference between high quality and low quality? From a  
high level perspective it’s the difference between real manual  
research based security testing or not. Once hackers have access, they  
can do anything to your data from steal it, to install back door  
technology in your product's source code. Its happened before, and its  
going to happen again.
When a company tells you that they perform manual testing hold their  
feet to the fire. You can do the following things to verify it:

	• Dig into their methodology and ask them specific questions about  
how they perform their testing. (See our white papers on how to do  
that).
	• Don’t swallow jargon and terms that sound cool and don’t mean  
anything, use Wikipedia to look up the terms and make sure that they  
make sense.
	• Ask them for the names of their security experts and then use tools  
like Google, LinkedIn, Facebook and PIPL to do research on those  
experts. If nothing comes up then chances are their experts aren’t  
experts at all.
	• Search vulnerability databases like milw0rm, securityfocus,sirtfr,  
secunia, packetstormsecurity, etc. for the vendor’s name to see if  
they have research capabilities. If you don’t get anything in return  
then chances are that they don’t have research capabilities. If that’s  
the case then how do you expect them to perform quality manual  
testing? Chances are that they won’t be able to.
Remember you’re putting the integrity of your business and its  
respective name into their hands.


        Adriel T. Desautels
        ad_lists () netragard com
        --------------------------------------

        Subscribe to our blog
        http://snosoft.blogspot.com


------------------------------------------------------------------------
This list is sponsored by: Information Assurance Certification Review Board

Prove to peers and potential employers without a doubt that you can actually do a proper penetration test. IACRB CPT 
and CEPT certs require a full practical examination in order to become certified.

http://www.iacertification.org
------------------------------------------------------------------------