Re: Integrating two solutions (related to the Calif. bill thread)

From: Richard C. Johnson <dick_at_iwwco_dot_com>
Date: Fri Jan 23 2009 - 10:45:52 CST

I have to more than agree with Ron, having done extensive coverage analysis. 100% coverage of any substantial software system is virtually impossible. My experience was that coverage analysis had to be supplemented by risk analysis of the remaining code, which for sure is not tested. Also, another aspect to coverage, one can cover or exercise code without putting it at any real risk of failure. That is, just because you exercise the code, you don't automatically assume that it has been tested. It is there for a reason, and coverage alone may not tell you whether its purpose is accomplished by your test.

Yes, you must do coverage analysis, but you must also be able to explain how given tests put code at risk of failure and what the risks are in the code you are unable (for whatever reason) to exercise. The task is non-trivial.

-- Dick Johnson

--- On Thu, 1/22/09, Ronald Crane <voting@lastland.net> wrote:

> From: Ronald Crane <voting@lastland.net>
> Subject: Re: [OVC-discuss] Integrating two solutions (related to the Calif. bill thread)
> To: "Open Voting Consortium discussion list" <ovc-discuss@listman.sonic.net>
> Date: Thursday, January 22, 2009, 2:11 PM
> Arthur Keller wrote:
> > At 8:03 PM -0800 1/21/09, Ronald Crane wrote:
> >> 4. Please see the "Limitations of Many
> Eyes" thread here, begun by Brian Behlendorf on
> 5/19/08, about a study by David Wagner & Ping Lee,
> showing code review's unexpectedly-limited efficacy in
> finding intentionally-placed security flaws. Presumably
> review is even less efficacious in the functionally-obscure,
> often highly-concurrent, and lower-level-language
> environments that usually characterize firmware.
> >
> > I'm wondering whether the approach in
> http://www.d50.org/ would have made a difference in the
> Wagner, et al., study.
> I'm not exactly sure what that site proposes, but it
> mentions "100% path coverage analysis". That's
> practical only for relatively small projects with limited
> concurrency. I'm not sure where the site is going with
> the concordance idea, since a crafty attacker would
> certainly avoid using any give-away terms in her attack
> code, and if she had to use them, she'd obscure them by
> encrypting them, then using a constant key to decrypt and
> display them at runtime.
>
> -R
>
> _______________________________________________
> OVC-discuss mailing list
> OVC-discuss@listman.sonic.net
> http://lists.sonic.net/mailman/listinfo/ovc-discuss
> By sending email to the OVC-discuss list, you thereby
> agree to release the content of your posts to the Public
> Domain--with the exception of copyrighted material quoted
> according to fair use, including publicly archiving at
> http://gnosis.python-hosting.com/voting-project/
_______________________________________________
OVC-discuss mailing list
OVC-discuss@listman.sonic.net
http://lists.sonic.net/mailman/listinfo/ovc-discuss
By sending email to the OVC-discuss list, you thereby agree to release the content of your posts to the Public Domain--with the exception of copyrighted material quoted according to fair use, including publicly archiving at http://gnosis.python-hosting.com/voting-project/
==================================================================
= The content of this message, with the exception of any external
= quotations under fair use, are released to the Public Domain
==================================================================
Received on Thu Jan 7 00:09:50 2010

This archive was generated by hypermail 2.1.8 : Thu Jan 07 2010 - 00:09:57 CST