Re: [OVC-discuss] Integrating two solutions (related to the Calif. bill thread)

From: Edward Cherlin <echerlin_at_gmail_dot_com>
Date: Thu Jan 22 2009 - 04:19:49 CST

On Thu, Jan 22, 2009 at 1:55 AM, Arthur Keller <voting@kellers.org> wrote:
> At 8:03 PM -0800 1/21/09, Ronald Crane wrote:
>>
>> 4. Please see the "Limitations of Many Eyes" thread here, begun by Brian
>> Behlendorf on 5/19/08, about a study by David Wagner & Ping Lee, showing
>> code review's unexpectedly-limited efficacy in finding intentionally-placed
>> security flaws.

That study draws much too broad conclusions from a very limited
experiment. It shows that the people chosen found only a certain
fraction of known (intentional) deficiencies in a body of code. This
says nothing about how code reviewers with appropriate skills and
training might do.

Many eyes are clearly not enough. Some of those eyes have to know what
they are looking for. I have had this experience in QA, where only
precise domain knowledge let me create appropriate test cases. You
need cryptographic experts, and experts in physical security,
including stage magicians, in order to have half a chance of catching
the most egregious security holes.

>> Presumably review is even less efficacious in the
>> functionally-obscure, often highly-concurrent, and lower-level-language
>> environments that usually characterize firmware.

This is a gross over-characterization, a stereotype, a prejudiced
parody of the truth. Firmware can be written in extremely obscure
ways, or in extremely transparent ways. I have direct experience of
both. Open FirmWare is the most transparent form of firmware I am
aware of. I know of thousands of people who can read it, and only a
few dozen who are at all comfortable reading the object code of a
BIOS. Peter Norton, for example, made his reputation on the basis of
such skills.

> I'm wondering whether the approach in http://www.d50.org/ would have made a
> difference in the Wagner, et al., study.
>
> Best regards,
> Arthur
>
> --
> -------------------------------------------------------------------------------
> Arthur M. Keller, Ph.D., 3881 Corina Way, Palo Alto, CA 94303-4507
> tel +1(650)424-0202, fax +1(650)424-0424
> _______________________________________________
> OVC-discuss mailing list
> OVC-discuss@listman.sonic.net
> http://lists.sonic.net/mailman/listinfo/ovc-discuss
> By sending email to the OVC-discuss list, you thereby agree to release the
> content of your posts to the Public Domain--with the exception of
> copyrighted material quoted according to fair use, including publicly
> archiving at http://gnosis.python-hosting.com/voting-project/
>

-- 
Silent Thunder (默雷/धर्ममेघशब्दगर्ज/دھرممیگھشبدگر ج) is my name
And Children are my nation.
The Cosmos is my dwelling place, The Truth my destination.
http://wiki.sugarlabs.org/go/User:Mokurai
_______________________________________________
OVC-discuss mailing list
OVC-discuss@listman.sonic.net
http://lists.sonic.net/mailman/listinfo/ovc-discuss
By sending email to the OVC-discuss  list, you thereby agree to release the content of your posts to the Public Domain--with the exception of copyrighted material quoted according to fair use, including publicly archiving at  http://gnosis.python-hosting.com/voting-project/
Received on Thu Jan 7 00:09:49 2010

This archive was generated by hypermail 2.1.8 : Thu Jan 07 2010 - 00:09:57 CST