Re: Proposed OVC Listed Policies

From: Richard C. Johnson <dick_at_iwwco_dot_com>
Date: Sun Dec 17 2006 - 19:29:30 CST

I like almost all of what Joe writes, except for the portion dealing with test policies and the ITAs. I think that this needs to be strengthened. First, there should be a standard for a minimum core of tests, all of which must be public along with the results of these tests. Second, states and/or the federal government must pay for any testing past this core minimum and, since the public paid for these tests, they should be public as well. (Note: the public, through its government, hired the ITA in this scenario and should own the output by contract.) Naturally, OVC members would comment on these tests and their adequacy.

Third, any additional testing vendors wish to do at their own expense may be private. Fourth, only public tests should determine certification. And, last but not least, all certification tests must be capable of being repeated for regression testing of any certified code a vendor modifies prior to use in an election.

Otherwise, I like Joe's notion of levels of OVC rating. OVS has products that include Oracle running on Linux where demands of a central election administration system require a sophisticated RDBMS; administration, however, is not voting and I can not see much reason for a voting system per se to include COTS software. Systems where the database or operating system are not open source (but the voting application is) could be classified separate from those having 100% open source software.

I would also hold open a category for voting systems in which the hardware components are Open Design (no proprietary hardware). It may be sooner than you think that such Open Designs are available and even the Open Voting Hardware to go with them.

-- Dick

Joseph Lorenzo Hall <> wrote: On 11/27/06, Arthur Keller wrote:
> At its October 20, 2006 meeting, the OVC Board voted unanimously to
> create "an OVC listed mark that is to be given solely on the nature
> of the disclosure, type of license, and vendor attestations." The
> OVC is to publish the disclosure, license, and vendor attestations on
> its websites. Details about the requirements of what is to be
> disclosed and under what licenses and what vendor attestations were
> to be posted to the OVC discuss list subsequently. Further
> discussions took place at the November 17, 2006 OVC Board meeting.
> The following proposal is based on these discussions.
> "OVC Listed" would say that the source, documentation, license, etc.,
> are publicly disclosed. The public disclosure of the source,
> documentation, license, etc., would be on our website. We need to
> come up with a checklist of the items that need to be disclosed. We
> would then provide a listing ID that one could look up on our website
> to have access to the publicly disclosed information.
> Submission components:
> 1. Listing of components of system (e.g., electronic ballot printer,
> ballot reconciliation system, ballot verification system) and their
> version identifiers.
> 2. Full source code for each voting-specific component of system,
> including make files, header files, and configuration files, and any
> other file needed to build a complete version of the system (given
> COTS object code).
> 3. Object code image for each component of system (including COTS components).
> 4. Checksums of object code image for each component of system (e.g.,
> MD5, etc.; list of required checksums to be updated by OVC from time
> to time).
> 5. Specifications.
> 6. Documentation.
> 7. Internal and external document formats and sample documents (e.g.,
> ballot definition files, cast ballot records, vote total records).
> 8. Hardware dependencies, specifications, and requirements.
> 9. For each Commercial-off-the-shelf (COTS) component (e.g.,
> operating system or device or printer drivers), specifications,
> version numbers, dates of manufacture. requirements and uses.
> 10. Feature checklist (e.g., paper ballot vs. electronic ballot with
> paper audit trail; basic architectural type).
> 11. License(s) for the system, as per the requirement below.
> 12. Reports of non-internal tests, such as by Independent Testing
> Authorities or acceptance tests by customers, as they are prepared.
> 13. An attestation that all components and descriptions submitted are
> accurate and represent the versions identified.
> The Vendor license must allow others also to publish the software,
> anyone to test and experiment and analyze, including publication of
> analyses including excerpts of source code. The vendor may retain
> copyrights, trademark rights, and patent rights, but not trade secret
> rights.
> OVC will assign an "OVC Listed" identifier, and publish all of the
> information submitted (on websites or through other means) with the
> identifier. Each configuration/version is a separate submission with
> its own "OVC Listed" identifier. Updated versions or configurations
> must be resubmitted to be OVC Listed.
> OVC will document the process for submitting a voting system to be OVC Listed.
> -------
> Note: Vendors are not required to grant the rights to make derivative
> works or derivative systems (other than as part of analyses, tests,
> or experiments), and the rights to use the system for elections would
> not be required to be granted as part of obtaining an OVC listing.
> Note: OVC may charge for listing or make iisting a benefit of certain
> classes of membership.
> Note: OVC Listed does not imply any testing of the voting system.
> The OVC will subsequently develop concepts of open testing,
> Note: The OVC may host a website or email list or other mechanism for
> reporting problems with voting systems (whether listed or not), or
> link to websites maintained by others.
> -------
> Note: Feedback on this proposal are requested. In particular,
> suggestions are requested on what should be in the feature checklist.

Hi everyone,

Here are a few initial comments:

* COTS source code - This proposal makes it clear that source code to
COTS software is not a requirement for OVC listing. Am I reading that
right? Have we fully thought out the implications -- both pro and con
-- of this choice? I realize that their are a number of goals
associated with source code disclosure including the potential for
improved security, reliability, etc.; however, not having access to
build-able COTS source provides a significant "openness sink". I
don't think anyone can successfully make the case that a system with
only COTS[1] object code available is less open than one that either
doesn't use third-party proprietary software or provides those sources
under a disclosure license that meets the requirements of OVC-listed.
I think there should be a few levels of listing... something like "all
components disclosed" and "voting system source disclosed".

[1] I should be precise: when I say COTS, I mean any third-party
proprietary software that a system uses.

* COTS licenses - Either way, when a submitter chooses to use either
discloseable or non-discloseable COTS software, there will be some
additional requirements that need to be placed on the licenses for
that software. For disclosed COTS software, It would seem that the
submitter would also need to negotiate an appropriate license with the
copyright owners of COTS software such that evaluators of the software
can make the same uses of the COTS source that they can of the voting
system source. For non-disclosed COTS software, there are certain
things that evaluators and others should specifically be allowed to
do. For example, any reverse-engineering clauses should not be
enforceable against licensees of the COTS object code in the voting
system evaluation context. That is, evaluators should be allowed to
engage in reverse-engineering of the COTS object code if, for example,
they trace an anomaly or other problem to the COTS component. There
are probably other parts of more traditional COTS EULA's that should
be unenforceable against voting system evaluators.

* Procurement contracts - It would be a small addition for
organizations with OVC-listed voting systems to also make available
the contracts that they execute with local jurisdictions. These are
typically already public but it would be good for them to provide the
contracts as opposed to members of the public having to submit
FOIA/PRA requests for them.

* Certification-related data - In addition to ITA/VSTL test reports
that are the product of certification at the federal level, there are
a few other things that would be good to explicitly collect. At the
federal level, all the materials in the Technical Data Package that
vendors submit to ITAs/VSTLs should be disclosed (this is input into
certification process). Also, the Test Plan that the ITA/VSTL uses to
evaluate systems should be disclosed[2]. At the state level, it would
seem very valuable to have the input, materials and output from tests
that the states feel are important to plug holes in the federal
process. For example, any material that is typically proprietary and
confidential used or created at the state certification level should
be examined and most likely disclosed (with each state having a
different certification process (or not having one at all) it's hard
to say what here would be valuable... the vendor would be in the best
position to judge this).

[2] This could be contentious. ITAs/VSTLs have maintained that their
test plans are proprietary. If a vendor seeking OVC listing seeks to
require an ITA/VSTL to allow the vendor to publicly disclose the test
plan used to evaluate their system, the ITA/VSTL may choose not to
take the business out of a fear that they may loose a competitive edge
in certification testing. I'm not sure to what extent the VSTLs that
have been approved by the EAC might choose not to do business with a
vendor that would seek to disclose test plans.

* Hardware specification - It's not clear from the outline of
submission components above to what level of detail hardware
specification is required. The EAC's VSTCP Manual seems to believe
that high-resolution images of hardware components is sufficient.
That's clearly a naive position. As there are people on this list
with much more experience than I have with respect to hardware, I'll
not discuss the requirements necessary for inspecting and possibly
verifying hardware designs (I realize that this is still an open area
of research).

* Hardware licenses - to the extent that a voting system uses custom
hardware (like chips) those should be disclosed with sufficient detail
to allow examination and verification as well as their designs
licensed under a license that is no more restrictive than the
companion software license.

Joseph Lorenzo Hall
PhD Student, UC Berkeley, School of Information
OVC-discuss mailing list

OVC-discuss mailing list

= The content of this message, with the exception of any external
= quotations under fair use, are released to the Public Domain
Received on Sun Dec 31 23:17:15 2006

This archive was generated by hypermail 2.1.8 : Sun Dec 31 2006 - 23:17:16 CST