Bugs and flaws

[mgavin at forrester.com (Gavin, Michael)]

"Architecture" is also an overloaded term, often meaning either a design
(the output of architects) or the implementation of certain types of
design (the output of engineers). 

Hoping to clarify Chris's comment on architecture flaws: architecture
defects as in the defects in the output produced by architects are
"design flaws"; architecture defects as in the defects in the output of
programmers/coders/engineers are "implementation flaws".

FWIW, I agree with Chris, "design flaw" and "implementation flaw" seem
better/more descriptive/less confusing than revised definitions for
"flaw" and "bug". (Then again, I once worked at @stake..., on the other
hand, IIRC this terminology is more consistent with what you find in
Ross Anderson's classic "Security Engineering".)

Michael

-----Original Message-----
From: sc-l-bounces at securecoding.org
[mailto:sc-l-bounces at securecoding.org] On Behalf Of Chris Wysopal
Sent: Thursday, February 02, 2006 4:35 PM
To: Gary McGraw
Cc: William Kruse; Wall, Kevin; Secure Coding Mailing List
Subject: RE: [SC-L] Bugs and flaws


In the manufacturing world, which is far more mature than the software
development world, they use the terminology of "design defect" and
"manufacturing defect".  So this distinction is useful and has stood the
test of time.

Flaw and defect are synonymous. We should just pick one. You could say
that the term for manufacturing software is "implementation".

So why do we need to change the terms for the software world?  Wouldn't
"design defect" and "implementation defect" be clearer and more in line
with the manufacturing quality discipline, which the software quality
discipline should be working towards emulating. (When do we get to Six
Sigma?)

I just don't see the usefulness of calling a "design defect" a "flaw".
"Flaw" by itself is overloaded.  And in the software world, "bug" can
mean
an implementation or design problem, so "bug" alone is overloaded for
describing an implementation defect.

At @stake the Application Center of Excellence used the terminology
"design flaw" and "implementation flaw".  It well understood by our
customers.

As Crispin said in an earlier post on the subject, the line is sometimes
blurry.  I am sure this is the case in manufacturing too.  Architecture
flaws can be folded into the design flaw category for simplicity.

My vote is for a less overloaded and clearer terminology.

-Chris

P.S. My father managed a non-destructive test lab at a jet engine
manufacturer. They had about the highest quality requirements in the
world. So for many hours I was regaled with tales about the benefits of
performing static analysis on individual components early in the
manufacturing cycle.

They would dip cast parts in a fluorescent liquid and look at them under
ultraviolet light to illuminate cracks caused during casting process.
For critical parts which would receive more stress, such as the fan
blades, they would x-ray each part to inspect for internal cracks. A
more
expensive process but warranted due to the increased risk of total
system
failure for a defect in those parts.

The static testing was obviously much cheaper and delivered better
quality
than just bolting the parts together and doing dynamic testing in a test
cell.  It's a wonder that it has taken the software security world so
long
to catch onto the benefits of static testing of implementation.  I think
we can learn a lot more from the manufacturing world.

On Thu, 2 Feb 2006, Gary McGraw wrote:

> Hi all,
>
> When I introduced the "bugs" and "flaws" nomenclature into the
> literature, I did so in an article about the software security
workshop
> I chaired in 2003 (see http://www.cigital.com/ssw/).  This was
> ultimately written up in an "On the Horizon" paper published by IEEE
> Security & Privacy.
>
> Nancy Mead and I queried the SWEBOK and looked around to see if the
new
> usage caused collision.  It did not.  The reason I think it is
important
> to distinguish the two ends of the rather slippery range (crispy is
> right about that) is that software security as a field is not paying
> enough attention to architecture.  By identifying flaws as a
subcategory
> of defects (according the the SWEBOK), we can focus some attention on
> the problem.
>
> > > From the small glossary in "Software Security" (my new book out
> tomorrow):
>
> Bug-A bug is an implementation-level software problem. Bugs may exist
in
> code but never be executed. Though the term bug is applied quite
> generally
> by many software practitioners, I reserve use of the term to encompass
> fairly
> simple implementation errors. Bugs are implementation-level problems
> that
> can be easily discovered and remedied. See Chapter 1.
>
> Flaw-A design-level or architectural software defect. High-level
defects
> cause 50% of software security problems. See Chapter 1.
>
> In any case, I intend to still use these terms like this, and I would
be
> very pleased if you would all join me.
>
> gem
------------------------------------------------------------------------
----
> This electronic message transmission contains information that may be
> confidential or privileged.  The information contained herein is
intended
> solely for the recipient and use by any other party is not authorized.
If
> you are not the intended recipient (or otherwise authorized to receive
this
> message by the intended recipient), any disclosure, copying,
distribution or
> use of the contents of the information is prohibited.  If you have
received
> this electronic message transmission in error, please contact the
sender by
> reply email and delete all copies of this message.  Cigital, Inc.
accepts no
> responsibility for any loss or damage resulting directly or indirectly
from
> the use of this email or its contents.
> Thank You.
------------------------------------------------------------------------
----
> _______________________________________________
> Secure Coding mailing list (SC-L)
> SC-L at securecoding.org
> List information, subscriptions, etc -
http://krvw.com/mailman/listinfo/sc-l
> List charter available at -
http://www.securecoding.org/list/charter.php
>
_______________________________________________
Secure Coding mailing list (SC-L)
SC-L at securecoding.org
List information, subscriptions, etc -
http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php