Re: Proposed OVC Listed Policies

From: Joseph Lorenzo Hall <joehall_at_pobox_dot_com>
Date: Mon Dec 18 2006 - 13:15:13 CST

On 12/17/06, Richard C. Johnson <> wrote:
> I like almost all of what Joe writes, except for the portion dealing with
> test policies and the ITAs. I think that this needs to be strengthened.
> First, there should be a standard for a minimum core of tests, all of which
> must be public along with the results of these tests. Second, states and/or
> the federal government must pay for any testing past this core minimum and,
> since the public paid for these tests, they should be public as well.
> (Note: the public, through its government, hired the ITA in this scenario
> and should own the output by contract.) Naturally, OVC members would
> comment on these tests and their adequacy.
> Third, any additional testing vendors wish to do at their own expense may be
> private. Fourth, only public tests should determine certification. And,
> last but not least, all certification tests must be capable of being
> repeated for regression testing of any certified code a vendor modifies
> prior to use in an election.

These are very good points and astute observations. I think there's a
little work between where we're at now and where we'd like to be to
conform to these principles. The first order of business seems to be
coming up with a testing regime for OVC-listed products. (Arthur, is
that out of scope for this first OVC-listed criteria effort?)

> Otherwise, I like Joe's notion of levels of OVC rating. OVS has products
> that include Oracle running on Linux where demands of a central election
> administration system require a sophisticated RDBMS; administration,
> however, is not voting and I can not see much reason for a voting system per
> se to include COTS software. Systems where the database or operating system
> are not open source (but the voting application is) could be classified
> separate from those having 100% open source software.
> I would also hold open a category for voting systems in which the hardware
> components are Open Design (no proprietary hardware). It may be sooner than
> you think that such Open Designs are available and even the Open Voting
> Hardware to go with them.
> -- Dick
> Joseph Lorenzo Hall <> wrote:
> On 11/27/06, Arthur Keller wrote:
> > At its October 20, 2006 meeting, the OVC Board voted unanimously to
> > create "an OVC listed mark that is to be given solely on the nature
> > of the disclosure, type of license, and vendor attestations." The
> > OVC is to publish the disclosure, license, and vendor attestations on
> > its websites. Details about the requirements of what is to be
> > disclosed and under what licenses and what vendor attestations were
> > to be posted to the OVC discuss list subsequently. Further
> > discussions took place at the November 17, 2006 OVC Board meeting.
> > The following proposal is based on these discussions.
> >
> > "OVC Listed" would say that the source, documentation, license, etc.,
> > are publicly disclosed. The public disclosure of the source,
> > documentation, license, etc., would be on our website. We need to
> > come up with a checklist of the items that need to be disclosed. We
> > would then provide a listing ID that one could look up on our website
> > to have access to the publicly disclosed information.
> >
> > Submission components:
> >
> > 1. Listing of components of system (e.g., electronic ballot printer,
> > ballot reconciliation system, ballot verification system) and their
> > version identifiers.
> > 2. Full source code for each voting-specific component of system,
> > including make files, header files, and configuration files, and any
> > other file needed to build a complete version of the system (given
> > COTS object code).
> > 3. Object code image for each component of system (including COTS
> components).
> > 4. Checksums of object code image for each component of system (e.g.,
> > MD5, etc.; list of required checksums to be updated by OVC from time
> > to time).
> > 5. Specifications.
> > 6. Documentation.
> > 7. Internal and external document formats and sample documents (e.g.,
> > ballot definition files, cast ballot records, vote total records).
> > 8. Hardware dependencies, specifications, and requirements.
> > 9. For each Commercial-off-the-shelf (COTS) component (e.g.,
> > operating system or device or printer drivers), specifications,
> > version numbers, dates of manufacture. requirements and uses.
> > 10. Feature checklist (e.g., paper ballot vs. electronic ballot with
> > paper audit trail; basic architectural type).
> > 11. License(s) for the system, as per the requirement below.
> > 12. Reports of non-internal tests, such as by Independent Testing
> > Authorities or acceptance tests by customers, as they are prepared.
> > 13. An attestation that all components and descriptions submitted are
> > accurate and represent the versions identified.
> >
> > The Vendor license must allow others also to publish the software,
> > anyone to test and experiment and analyze, including publication of
> > analyses including excerpts of source code. The vendor may retain
> > copyrights, trademark rights, and patent rights, but not trade secret
> > rights.
> >
> > OVC will assign an "OVC Listed" identifier, and publish all of the
> > information submitted (on websites or through other means) with the
> > identifier. Each configuration/version is a separate submission with
> > its own "OVC Listed" identifier. Updated versions or configurations
> > must be resubmitted to be OVC Listed.
> >
> > OVC will document the process for submitting a voting system to be OVC
> Listed.
> >
> > -------
> >
> > Note: Vendors are not required to grant the rights to make derivative
> > works or derivative systems (other than as part of analyses, tests,
> > or experiments), and the rights to use the system for elections would
> > not be required to be granted as part of obtaining an OVC listing.
> >
> > Note: OVC may charge for listing or make iisting a benefit of certain
> > classes of membership.
> >
> > Note: OVC Listed does not imply any testing of the voting system.
> > The OVC will subsequently develop concepts of open testing,
> >
> > Note: The OVC may host a website or email list or other mechanism for
> > reporting problems with voting systems (whether listed or not), or
> > link to websites maintained by others.
> >
> > -------
> >
> > Note: Feedback on this proposal are requested. In particular,
> > suggestions are requested on what should be in the feature checklist.
> Hi everyone,
> Here are a few initial comments:
> * COTS source code - This proposal makes it clear that source code to
> COTS software is not a requirement for OVC listing. Am I reading that
> right? Have we fully thought out the implications -- both pro and con
> -- of this choice? I realize that their are a number of goals
> associated with source code disclosure including the potential for
> improved security, reliability, etc.; however, not having access to
> build-able COTS source provides a significant "openness sink". I
> don't think anyone can successfully make the case that a system with
> only COTS[1] object code available is less open than one that either
> doesn't use third-party proprietary software or provides those sources
> under a disclosure license that meets the requirements of OVC-listed.
> I think there should be a few levels of listing... something like "all
> components disclosed" and "voting system source disclosed".
> [1] I should be precise: when I say COTS, I mean any third-party
> proprietary software that a system uses.
> * COTS licenses - Either way, when a submitter chooses to use either
> discloseable or non-discloseable COTS software, there will be some
> additional requirements that need to be placed on the licenses for
> that software. For disclosed COTS software, It would seem that the
> submitter would also need to negotiate an appropriate license with the
> copyright owners of COTS software such that evaluators of the software
> can make the same uses of the COTS source that they can of the voting
> system source. For non-disclosed COTS software, there are certain
> things that evaluators and others should specifically be allowed to
> do. For example, any reverse-engineering clauses should not be
> enforceable against licensees of the COTS object code in the voting
> system evaluation context. That is, evaluators should be allowed to
> engage in reverse-engineering of the COTS object code if, for example,
> they trace an anomaly or other problem to the COTS component. There
> are probably other parts of more traditional COTS EULA's that should
> be unenforceable against voting system evaluators.
> * Procurement contracts - It would be a small addition for
> organizations with OVC-listed voting systems to also make available
> the contracts that they execute with local jurisdictions. These are
> typically already public but it would be good for them to provide the
> contracts as opposed to members of the public having to submit
> FOIA/PRA requests for them.
> * Certification-related data - In addition to ITA/VSTL test reports
> that are the product of certification at the federal level, there are
> a few other things that would be good to explicitly collect. At the
> federal level, all the materials in the Technical Data Package that
> vendors submit to ITAs/VSTLs should be disclosed (this is input into
> certification process). Also, the Test Plan that the ITA/VSTL uses to
> evaluate systems should be disclosed[2]. At the state level, it would
> seem very valuable to have the input, materials and output from tests
> that the states feel are important to plug holes in the federal
> process. For example, any material that is typically proprietary and
> confidential used or created at the state certification level should
> be examined and most likely disclosed (with each state having a
> different certification process (or not having one at all) it's hard
> to say what here would be valuable... the vendor would be in the best
> position to judge this).
> [2] This could be contentious. ITAs/VSTLs have maintained that their
> test plans are proprietary. If a vendor seeking OVC listing seeks to
> require an ITA/VSTL to allow the vendor to publicly disclose the test
> plan used to evaluate their system, the ITA/VSTL may choose not to
> take the business out of a fear that they may loose a competitive edge
> in certification testing. I'm not sure to what extent the VSTLs that
> have been approved by the EAC might choose not to do business with a
> vendor that would seek to disclose test plans.
> * Hardware specification - It's not clear from the outline of
> submission components above to what level of detail hardware
> specification is required. The EAC's VSTCP Manual seems to believe
> that high-resolution images of hardware components is sufficient.
> That's clearly a naive position. As there are people on this list
> with much more experience than I have with respect to hardware, I'll
> not discuss the requirements necessary for inspecting and possibly
> verifying hardware designs (I realize that this is still an open area
> of research).
> * Hardware licenses - to the extent that a voting system uses custom
> hardware (like chips) those should be disclosed with sufficient detail
> to allow examination and verification as well as their designs
> licensed under a license that is no more restrictive than the
> companion software license.
> --
> Joseph Lorenzo Hall
> PhD Student, UC Berkeley, School of Information
> _______________________________________________
> OVC-discuss mailing list
> _______________________________________________
> OVC-discuss mailing list

Joseph Lorenzo Hall
PhD Student, UC Berkeley, School of Information
OVC-discuss mailing list
= The content of this message, with the exception of any external 
= quotations under fair use, are released to the Public Domain    
Received on Sun Dec 31 23:17:15 2006

This archive was generated by hypermail 2.1.8 : Sun Dec 31 2006 - 23:17:16 CST