Re: Penetration testing via shrinkware

[Dave Whitlow <dwhitlow () wend dircon co uk>]

On Fri, 18 Sep 1998, Crispin Cowan wrote:

> tqbf () pobox com wrote:
>
> > > person/company for the job... Problem is, which tools and which people do
> > > you trust? Sounds like the subject of certification and accreditation comes
> > > back into play...
> >
> > Scanners are probably much easier to certify than firewalls (which
> > probably can't be meaningfully certified at all).

> I beg to differ.  A firewall can at least theoretically be verified:  if it is
> formally proven to enforce a policy of (say) allowing through traffic on ports X
> and Y, and no others, then the firewall is verified.  A scanner, on the other
> hand, can never be verified, because the potential list of vulnerabilities that
> it could reasonably be expected to check for is infinite.  Scanners can never be
> complete, because the space of possible mis-configurations and buggy software
> knows no bounds.

Not so.  A scanner performs a clearly defined (and mercifully enumerated) 
list of checks.  Each check is designed to demonstrate either the presence
(or absence) of a known security weakness or feature which the scanner
developers deem to be undesirable.  The scanner producers do not claim
that it does anything more than identify potential weaknesses.  In other
words it is easy to verify by either examining the packet stream for the
correct behavior or by offering systems with the conditions it claims to
check for.

Firewalls by their nature claim to protect one network from another,
usually untrusted, network.  Most modern commercial firewalls are complex
(what happened to the idea of simplicity in firewalls ?) and vulnerable to
exploitation of: 

a) programming (or design) errors within the firewall code 
b) programming (or design) errors within the underlying OS
c) hardware and firmware bugs
d) mis-configuration
e) new ways of abusing IP services for which the firewall offers no
defense.

Also, a particular release of a scanner can be tested whereas a firewall
can only be shown to provide a particular level of protection at a moment
in time.  Any change to the underlying OS, the configuration of discovery
of new exploits changes the level of trust that can be placed on the
firewall.


Dave
--
Dave Whitlow
EMail: dwhitlow () wend dircon co uk