Home page logo

pen-test logo Penetration Testing mailing list archives

Re: [PEN-TEST] Evaluating Auditors Abilities
From: Steve <steve () SECURESOLUTIONS ORG>
Date: Thu, 7 Sep 2000 10:33:47 -0600

Hash: SHA1

I hate to say it but I agree with some of Derrick's findings.  In a
past employment situation, I found the company (I won't name them but
they are a big 5) that we used for IT Security Audits spend more
time/money on things that turned out to be non-issues.  I even saw
one case where some "kid" walked into a banking client armed with a
copy of a popular scanning tool, ran the scan, printed the report and
handed it over along with a bill.  Ask him to explain the report of
the false positives and he couldn't.

Its clear that companies need to do their homework before hiring
someone to do an audit.  I would recommend looking at the potential
auditor as a whole, who they employ, and what discoveries/advisories
they have released.  See if they can provide references and don't be
fooled by the marketing machines of the bigger companies.  Some of
the most talented people work for smaller organizations.


Steve Manzuik
Moderator - Win2K Security Advice

Security Analyst - Bindview RAZOR

- -------------------------------------------

-----Original Message-----
From: Penetration Testers [mailto:PEN-TEST () SECURITYFOCUS COM]On
Behalf Of Derrick
Sent: Wednesday, September 06, 2000 10:46 PM
Subject: [PEN-TEST] Evaluating Auditors Abilities

Dear Pen-Testers,

      Recently I underwent something that had me thinking about
Security Auditing
companies and others (Big accounting firms that offer a side
service of auditing). Management decided that we needed to be
audited by an outside firm, which I am in full favor of. The
problem came about in what an un-named auditor did. Firewalls tend
to cause false positives in
some tests
and other anomalies that many auditors may not be aware of. So they
performed this audit which we did pick up and were aware of. What
happened next is what baffles me. The auditors did not understand
the results that nmap and other tools gave them. Near the end of
the business day they contact management proclaiming they have
found numerous security
issues and
even some backdoors in our network. After a long couple of days of
testing we found none of these issues were correct, and we then
many hours and
several meetings explaining that the firm hired didn't seem to know
what they were doing. Management made the default comment of "We
paying them
a lot so they must be right, fix these problems". After several
days of explaining why they results were wrong and verifying the
we came out
to show that the auditors did in fact improperly interpret the
results.      The end result is management walks away wondering if they
got ripped off or
if we were just trying to cover problems. It also caused a lot of
overtime and extra work for us to explain and prove the network to
management. So the
end questions are these.

How can companies decide which auditors really do a decent job
and are worth
their value ?
Are there any certifications or Industry groups out there or on
the horizon
that will evaluate and endorse auditors ?
What is the best approach from a Network Admin position to counter
end results delivered by auditors if they seem to be in error ?
Has anyone else been through this, and is destined to get worse
before getting better ?

Thanks for any thoughts or comments,

Version: PGPfreeware 6.5.3 for non-commercial use <http://www.pgp.com>


  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]