Home page logo
/

basics logo Security Basics mailing list archives

RE: How does a customer get PCI audited?
From: Craig Wright <Craig.Wright () bdo com au>
Date: Thu, 5 Jun 2008 13:17:46 +1000


Hi Erin,
I would disagree. I would split off "compliance" and "perception of compliance". Passing an audit is evidence that a 
system could be compliant. A compromise of a system using a know vulnerability is strong evidence that it is not 
compliant.

A system that is breached due to a complex password and secured key that was "guessed" is possible, though unlikely. 
This would one of the few examples of a compliant system that is also breached. Basically, it will be rare to find a 
compliant system (to any jurisdiction) that is easily compromised.

What I learnt completing my LLM was how few systems are complaint. How little knowledge there is of the law and legal 
frameworks already in place (even with politicians) and the lack of due care. An ABSOLUTE baseline for a compliant 
system that has not other effect other than being owned by a company would be the CISecurity.org baselines at 100%.

The combination of technical people with no knowledge of the legal system, laws and processes with lawyers who can not 
turn on a PC is an issue here.

"I would agree with Adriel that finding a worthwhile auditor is difficult". Actually so would I. Finding an staff with 
half a brain provides enough difficulty to want to give up on the whole idea.

"The problems are analyzed from a primarily financial and business risk avoidance perspective"
Here I have to disagree. I work with financial auditors and I am yet to meet one who understands risk and have met very 
few who have the faintest comprehension of finance. Audit and finance are NOT the same thing. I did finance at a 
masters level and I think audit is wacky for the most part. For the rest, there is an approach of try to find nothing 
wrong or it will upset the client.

I have developed statistically based continuous audit programs for financial systems. These have a significantly lower 
cost and deliver more. What I get back is "Craig, we are watch dogs and not blood hounds. Please try not to find so 
much". So I use these with the Insolvency teams and on forensic audits, but it is a hard sell to audit teams. Clients 
seem to love it though.

"I'm curious as to what vulnerable points you're thinking of." Pen Testing is by nature externally focused. Many of the 
biggest issues are in the system. Static analysis of code, business process reviews and system walkthroughs all add 
additional layers of testing.

Many controls are not tested using Pen testing ion any effective manner. Take a banking application. Pen tests look at 
the system from a software and protocol implementation aspect. They do not go into the business process controls. In 
this case I would be asking, how do I get the money off the system. This requires an understanding of the controls in 
the application. This is not something a pen test will provide.

An attacker can do this by compromising the system and modelling the application functions. Or the attacker could be 
internal and know them. This takes time, it can take months or longer. The same process can be done in a matter of 
weeks with a cryptal box and business process approach. The pen test provides valuable information as to a known 
vulnerability, but this is where it stops.

When there are no obvious points of access that may be exploited, a pen test does nothing to state a system is secure, 
just that it failed to determine the state of the system and was unable to determine if a system was secure or not.

Prof. Cohen developed the concept of protection testing over a decade ago. This mitigates many of the problems with a 
pen test methodology (that where noted as far back as 1977 by Distraka). The issue is that the tester needs more 
knowledge than a pen tester.

Protection testing really requires a combination of technical and business process skills. Teams can do this, but this 
increases cost and also requires a co-ordination factor with knowledge.

"what if the economics aspect were ignored"
Not my world. We live in a world where EVERTHING is subject to economic constraints and interrelationships.

Time for instance is a economic constraint. They only way to remove it is to have an instantaneous pen test. A 
detective control that reports faster than a pen test is more effective. The lengthy of the pen test is one factor, but 
also the frequency. 1 test a quarter is detection every 3 months at best.

Regards,
Craig Wright GSE LLM ...


Craig Wright
Manager, Risk Advisory Services

Direct : +61 2 9286 5497
Craig.Wright () bdo com au
+61 417 683 914

BDO Kendalls (NSW-VIC) Pty. Ltd.
Level 19, 2 Market Street Sydney NSW 2000
GPO BOX 2551 Sydney NSW 2001
Fax +61 2 9993 9497
http://www.bdo.com.au/

The information in this email and any attachments is confidential. If you are not the named addressee you must not 
read, print, copy, distribute, or use in any way this transmission or any information it contains. If you have received 
this message in error, please notify the sender by return email, destroy all copies and delete it from your system.

Any views expressed in this message are those of the individual sender and not necessarily endorsed by BDO Kendalls. 
You may not rely on this message as advice unless subsequently confirmed by fax or letter signed by a Partner or 
Director of BDO Kendalls. It is your responsibility to scan this communication and any files attached for computer 
viruses and other defects. BDO Kendalls does not accept liability for any loss or damage however caused which may 
result from this communication or any files attached. A full version of the BDO Kendalls disclaimer, and our Privacy 
statement, can be found on the BDO Kendalls website at http://www.bdo.com.au/ or by emailing mailto:administrator () 
bdo com au 

BDO Kendalls is a national association of separate partnerships and entities. Liability limited by a scheme approved 
under Professional Standards Legislation.
-----Original Message-----

From: Erin Carroll [mailto:amoeba () amoebazone com]
Sent: Thursday, 5 June 2008 3:24 AM
To: 'Adriel Desautels'; Craig Wright
Cc: 'Scott Race'; security-basics () securityfocus com
Subject: RE: How does a customer get PCI audited?

Very interesting points from both of you that I'd like to respond to and
follow up on though it does range farther away from the Scott's original
question.

I think we can agree that standards and compliance controls only provide a
functional framework for potential security but are by no means a valid
indicator of actual security. Security Compliance isn't security, just
compliance. It's easy to become PCI compliant without increasing your
overall real-world security defense but PCI DSS is one of the better
standards in that the control guidelines map to actual security fairly well.
It's one of the few widely adhered-to standards I can point to where this is
the case the majority of time, though focused on a particular aspect (CC
info).

I would agree with Adriel that finding a worthwhile auditor is difficult but
I believe it's a problem endemic to the compliance and audit mindset in
general. The problems are analyzed from a primarily financial and business
risk avoidance perspective, not security, so increased real-world security
is just a side effect of the compliance process. Until security is viewed as
something other than a cost center I don't see the auditor services industry
making a push away from the "scan & report" practice. IMHO, QSA's whose
primary business is the technical security services (pen-testing, VA, etc)
as opposed to auditing will provide the best ROI.

One thing that I found interesting was Craig's statement that "Pen testing
is about a 30% option that ignores many of the vulnerable points of a
system." I'm curious as to what vulnerable points you're thinking of. I
agree that pen-testing doesn't address all vulnerability aspects but maybe
you could provide some examples for context. You say that economic
constraints limit it and that pen-testing isn't the best bang-for-the-buck
detective control but what if the economics aspect were ignored? Some idea
of the effectiveness of each detective control (VA, pen-test, policy,
process, etc) as applied to overall security to assign a weight value (minus
cost) would level the discussion playing field to debate the merits of each
aspect. Obviously cost changes those weight values but you'd get a sense of
how big a slice of the pie chart for each and can more intelligently
determine a ROI for securing your systems/business. A 20% option at $10
isn't as cost efficient as a 30% option at $15.

I have yet to read all the way through Craig's paper but I'm working on it
:)


--
Erin Carroll
Moderator, SecurityFocus pen-test mailing list
amoeba () amoebazone com
"Do Not Taunt Happy-Fun Ball"






-----Original Message-----
From: listbounce () securityfocus com [mailto:listbounce () securityfocus com] On
Behalf Of Adriel Desautels
Sent: Wednesday, June 04, 2008 8:08 AM
To: Craig Wright
Cc: Scott Race; security-basics () securityfocus com
Subject: Re: How does a customer get PCI audited?

Craig,
        I'm afraid that you didn't read what I wrote carefully enough or
that I
wasn't sufficiently clear in my communications. If you go back and read
what I said, I explicitly used the words "real" and "good".

        You can be purple in the face with controls and training, but if you

are never PROPERLY tested by a REAL team then you will never know where
your REAL risks are. The end solution consists of the PROPER
implementation of controls mixed with the right amount of training. You
can't create a GOOD solution without knowing details about the problem.

        The unfortunate truth is that MOST businesses that provide
"auditing"
type services, or assessing type services do not know what they are
doing. In fact, very sadly most of them run automated scanners and
produce deliverables that are the product of those scanners. Those
deliverables should be free in my opinion as they are usually poor quality.

        In order to properly defend a network you must first know what you
need
to defend it against. You must have a strong understanding of the threat
and how the threat might align with your risk and exposure profile. The
only way to do that is to either have good threat intelligence, or work
with a qualified penetration testing team that has REAL threat
intelligence.

        Once you've identified such a firm your IT Infrastructure,
personnel,
policies, etc. need to be tested at the same or higher threat level as
you'd face in the real world. That will identify your risks and help you
to build the proper CONTROLS to counter those risks. Suggesting that
anyone build controls without first having a GOOD and REAL assessment is
horrible advice. That would be akin to building defenses against Russia
during the cold war with no intelligence about their capabilities.

        With respect to your paper, I still need to go read it. That said,
even
if Penetration Testing is 30% of the total solution, it is clearly the
foundation to building the solution. Else you are building a blind
defense that most probably won't work.

        Its common sense Craig, know your enemy, know yourself, and then you

can build a good defense.

Regards,
        Adriel T. Desautels
        Chief Technology Officer
        Netragard, LLC.
        Office : 617-934-0269
        Mobile : 617-633-3821
        http://www.linkedin.com/pub/1/118/a45

        Join the Netragard, LLC. Linked In Group:
        http://www.linkedin.com/e/gis/48683/0B98E1705142

---------------------------------------------------------------
Netragard, LLC - http://www.netragard.com  -  "We make IT Safe"
Penetration Testing, Vulnerability Assessments, Website Security

Netragard Whitepaper Downloads:
-------------------------------
Choosing the right provider : http://tinyurl.com/2ahk3j
Three Things you must know  : http://tinyurl.com/26pjsn


Craig Wright wrote:
"You secure your systems by having an IT Security Company perform a real
good assessment"
No, good controls and training help secure a system. Pen testing is a
detective control. No more. In fact, it is not even close to being the best
(bang for buck) detective control.

Actually, Pen testing is about a 30% option that ignores many of the
vulnerable points of a system. Economic limits constrain it. The difficulty

I did some scientific research into this in the past. I published in the
IIA journal and a few others and a paper is on the sans reading room as
well:
http://www.sans.org/reading_room/whitepapers/auditing/1801.php

The difficultly is that there are very few auditors with a deep technical
level of knowledge.

Craig


Craig Wright
Manager, Risk Advisory Services

Direct : +61 2 9286 5497
Craig.Wright () bdo com au
+61 417 683 914

BDO Kendalls (NSW-VIC) Pty. Ltd.
Level 19, 2 Market Street Sydney NSW 2000
GPO BOX 2551 Sydney NSW 2001
Fax +61 2 9993 9497
http://www.bdo.com.au/

The information in this email and any attachments is confidential. If you
are not the named addressee you must not read, print, copy, distribute, or
use in any way this transmission or any information it contains. If you have
received this message in error, please notify the sender by return email,
destroy all copies and delete it from your system.

Any views expressed in this message are those of the individual sender and
not necessarily endorsed by BDO Kendalls. You may not rely on this message
as advice unless subsequently confirmed by fax or letter signed by a Partner
or Director of BDO Kendalls. It is your responsibility to scan this
communication and any files attached for computer viruses and other defects.
BDO Kendalls does not accept liability for any loss or damage however caused
which may result from this communication or any files attached. A full
version of the BDO Kendalls disclaimer, and our Privacy statement, can be
found on the BDO Kendalls website at http://www.bdo.com.au/ or by emailing
mailto:administrator () bdo com au 

BDO Kendalls is a national association of separate partnerships and
entities. Liability limited by a scheme approved under Professional
Standards Legislation.
-----Original Message-----

From: listbounce () securityfocus com [mailto:listbounce () securityfocus com]
On Behalf Of Adriel Desautels
Sent: Wednesday, 4 June 2008 7:25 AM
To: Craig Wright
Cc: Scott Race; security-basics () securityfocus com
Subject: Re: How does a customer get PCI audited?

Craig,
        You secure your systems by having an IT Security Company perform a
real
good assessment. It should be either a Penetration Test or a
Vulnerability Assessment. Penetration Tests are more in-depth,
Vulnerability Assessments are more safe. If you want information on both
of those services I've got a few white-papers that we've written that
you can download. Unfortunately, it is very hard to find good vendors
these days.

Regards,
        Adriel T. Desautels
        Chief Technology Officer
        Netragard, LLC.
        Office : 617-934-0269
        Mobile : 617-633-3821
        http://www.linkedin.com/pub/1/118/a45

        Join the Netragard, LLC. Linked In Group:
        http://www.linkedin.com/e/gis/48683/0B98E1705142

---------------------------------------------------------------
Netragard, LLC - http://www.netragard.com  -  "We make IT Safe"
Penetration Testing, Vulnerability Assessments, Website Security

Netragard Whitepaper Downloads:
-------------------------------
Choosing the right provider : http://tinyurl.com/2ahk3j
Three Things you must know  : http://tinyurl.com/26pjsn


Craig Wright wrote:
"What are the chances of them getting audited?"
PCI Top level = 100%
A break-in occurs = 99.5%

A better question is how do I pass an audit and better yet, how do I
secure my systems.

PCI-DSS is just the tip of the iceberg for potential issues. In fact it
is the low risk consideration. Think class action. PCI-DSS compliance is at
least a small defence. That is being ACTUALLY compliant and not just doing
enough to fool the QSA.

Regards,
CSW GSE LLM


Craig Wright
Manager, Risk Advisory Services

Direct : +61 2 9286 5497
Craig.Wright () bdo com au
+61 417 683 914

BDO Kendalls (NSW-VIC) Pty. Ltd.
Level 19, 2 Market Street Sydney NSW 2000
GPO BOX 2551 Sydney NSW 2001
Fax +61 2 9993 9497
http://www.bdo.com.au/

The information in this email and any attachments is confidential. If you
are not the named addressee you must not read, print, copy, distribute, or
use in any way this transmission or any information it contains. If you have
received this message in error, please notify the sender by return email,
destroy all copies and delete it from your system.

Any views expressed in this message are those of the individual sender
and not necessarily endorsed by BDO Kendalls. You may not rely on this
message as advice unless subsequently confirmed by fax or letter signed by a
Partner or Director of BDO Kendalls. It is your responsibility to scan this
communication and any files attached for computer viruses and other defects.
BDO Kendalls does not accept liability for any loss or damage however caused
which may result from this communication or any files attached. A full
version of the BDO Kendalls disclaimer, and our Privacy statement, can be
found on the BDO Kendalls website at http://www.bdo.com.au/ or by emailing
mailto:administrator () bdo com au 

BDO Kendalls is a national association of separate partnerships and
entities. Liability limited by a scheme approved under Professional
Standards Legislation.
-----Original Message-----

From: listbounce () securityfocus com [mailto:listbounce () securityfocus com]
On Behalf Of Scott Race
Sent: Tuesday, 3 June 2008 8:37 AM
To: security-basics () securityfocus com
Subject: How does a customer get PCI audited?

I have a client (same one from a previous post) who has some pretty
serious security issues on their network (unsecured .mdb file with
credit card into, etc).  I will be fixing the major security holes in
their network, but they still have PCI compliance issues, and I'm
assuming they need to have a quarterly scan done.

They've had this setup for about a year, they knows nothing about PCI
and compliance (myself included, I am not a QSA and still learning about
the compliance procedure).

What are the chances of them getting audited?  How does all that work?
Could they potentially fly under the radar for years?  I thought there
was something they had to report quarterly to show they're working on
compliance, or something.

I want to be able to tell they company "Listen, here's what could happen
if you get audited, and here's the chances of you getting audited" in
hopes they would take it seriously.  I don't want to scare them without
knowing the facts, first I want to know the facts, then I will scare
them.  Thanks.
Scott Race
Technology Manager

JD+A NETWORK SERVICES
1264 Hawks Flight Court, Suite 200

El Dorado Hills, CA 95762
P:  916.941.3700  |  F:  916.941.3777


ethingHo????




  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]