Home page logo

pen-test logo Penetration Testing mailing list archives

Re: What a security test should do?- from thinking about: Ethical Hacking Training
From: "James Fields" <jvfields () tds net>
Date: Fri, 23 Jan 2004 17:20:31 -0500

Agreed.  My company has purchased testing from an extremely well-known and
respected firm.  The tests are minimally useful in pointing out an
occassional flaw or a missed configuration step on a web server.  However,
the reports are not detailed at all, and fail to give an accounting of
everything that was tested.  This is important to me and my company - we pay
a LOT of money for these tests, and not knowing what was tested leaves us
blind as to what we're really paying for.  It certainly isn't a trophy hunt.
I personally need to know all that was tried, and when, so that I can
compare the test activity to my firewall and IDS logs and see if I logged
all that I should have, or if I interpreted correctly what was being done
during the test.  It's one thing I really like about the OSSTMM - you have
to lay it all out, whether you successfully exploit anything or not.
Full-disclosure is good for the tester...

----- Original Message -----
From: "Pete Herzog" <pete () isecom org>
To: "Jeff Shawgo" <jeff.shawgo () verizon net>; <pen-test () securityfocus com>
Sent: Friday, January 23, 2004 3:32 PM
Subject: What a security test should do?- from thinking about: Ethical
Hacking Training

What does a pen test fail to provide?

I had to think about this for a little while because it's not so much to
what someone needs to know to be a security manager, CISO, or security
consultant, but rather what do we expect from a security test?

I know what pen-tests have been used for but I think a lot of that is also
under-analyzing the results of a pen-tset.  As an auditor of pen-test
reports for some companies, I see many of these reports focusing on
vulnerabilities, the occassional rooting of boxes, and the holy trilogy of
web app hacks (XSS, Command Injection, Buffer Overflows).  Most reports
have a traceroute to each host in the network but not even say why or what
that is useful for.  So in the end these reports leave a lot of analysis
to the client and if they are not capable of this kind of analysis, the
report has much less worth.

I have felt that security tests should do more. They should test
configurations and policies as well.  A test may tell you, for example,
about patch management, which department influences the company's Internet
presence, and if the firewall admin has top-level support or a policy to
follow regarding opening new ports.  All of these things may negatively
influence the strength of network security in ways that make it just as
vulnerable as a remote service exploit.

As Jeff mentions here, there is a lot more to network security than
pen-testing but for the most part, testing should be also able to verify
when the foundation is rotten.

So my question is, what parts of security can't be verified in a security
test?  No flames please-- I'm just trying to make the OSSTMM (osstmm.org)


Pete Herzog, Managing Director
Institute for Security and Open Methodologies
www.isecom.org - www.osstmm.org
www.hackerhighschool.org - www.isestorm.org

-----Original Message-----
From: Jeff Shawgo [mailto:jeff.shawgo () verizon net]
Sent: Tuesday, January 20, 2004 18:46 PM
To: pen-test () securityfocus com
Subject: Re: Ethical Hacking Training

On the other hand, most people also forget that knowing how to
perform a pen-test or exploit is only one very very tiny aspect
of security.  The organization that has a solid policy,
coordinated antivirus, well-managed firewalls, patch management
policy, e-mail and web filtering, code review, and basic system
hardening is likely to be many times more secure than the
organization that focuses on *any* one individual's skill as a

If the security foundation is rotten, it does little good to
point out that the windows are unlocked.

Pen-testing is important, but the basics need to be there first.
That's the message most people are missing - probably because
it's not as attractive.





  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]