Re: Open Source Quality

From: David Mertz <voting-project_at_gnosis_dot_cx>
Date: Sun May 09 2004 - 12:49:01 CDT

On May 9, 2004, at 12:57 PM, Douglas W. Jones wrote:
> Fuzz Revisited: A Re-examination of the Reliability of UNIX Utilities
> and Services
> http://www.opensource.org/advocacy/fuzz-revisited.pdf

I particularly like:

> The presence of these errors argues (minimally) for garbage-collected
> languages and fulltime array bounds checking to help a sloppy
> programmer detect these problems.

I.e. Python is the *right* language to write OVC's reference in.

Most of the others aren't things the language solves for us, though
there are right ways to handle them in Python (and more easily than in
languages like C/C++, or even Java).

Overall, it's quite striking how much more reliable GNU utilities are
than commercial equivalents.

However, the paper made a peculiar error in its description. It
described the bugs detected as including "infinite loops." Well,
there's this little thing called the Halting Problem that says you
don't know in the general case if a program will ever end. Given that
the "Fuzz Tests" are strictly behavior, rather than by code path
inspection, the testers have no way of knowing for sure that a given
program wouldn't have stopped two seconds after they canceled the test
run, all by itself. Operations that are *long* are not necessarily
infinite.

I recognize that it is reasonable to expect operations to perform in
*reasonable* time frames. If some utility finishes almost all inputs
inside one second, then takes a month working on a superficially
similar input, we are probably warranted in calling that a bug. Not
100%, there are some math oddities I could come up with, where some
operation really does just take a month to calculate as an algorithmic
requirement (even though most other calculations take a second). But
you don't want, e.g. 'grep' to spend a month searching a given file, or
on a given regex. But still... the authors don't *know* they found
infinite loops, which is what the article claims, merely that they
found overly slow code paths.
==================================================================
= The content of this message, with the exception of any external
= quotations under fair use, are released to the Public Domain
==================================================================
Received on Mon May 31 23:17:27 2004

This archive was generated by hypermail 2.1.8 : Mon May 31 2004 - 23:18:16 CDT