Shamos rebuttal -- re-visit

From: Alan Dechert <dechert_at_gmail_dot_com>
Date: Tue Apr 12 2005 - 16:51:36 CDT

I wrote the following last year to the voting-project list. I was
hoping to get some good input but for some reason I got very little
feedback. I want to re-visit this because someone wanting to rebut
Shamos found this article and wants to submit it to officials in PA.

Please, if some of you could sharpen your pencils and go over this,
I'd appreciate it.

Alan D.

********
OVC Response to Paper v. Electronic Voting Records -- An Assessment,
by Michael Ian Shamos
This message: [ Message body ] [ More options ]
Related messages: [ Next message ] [ Previous message ]
From: Alan Dechert <alan_at_openvotingconsortium_dot_org>
Date: Fri Jul 30 2004 - 22:11:03 CDT

Michael Ian Shamos is often quoted by individuals and organizations that
want to say DREs are the way to go. He is a highly credible individual, so
what he says on this subject matters a great deal.

The more I look at it, the more his arguments seem incredibly bad. David
Jefferson, a friend of Shamos for 30 years, has said that he is doing a lot
of damage and his POV on DREs should be discredited.

Here is a paper he wrote for the CFP 2004 conference we were at.

http://euro.ecom.cmu.edu/people/faculty/mshamos/paper.htm#_edn1

I have made a rough first cut at a response. I'd like to go through a few
iterations on this and then distribute it to the press and other interested
parties.

*************
OVC Response to Paper v. Electronic Voting Records -- An Assessment, by
Michael Ian Shamos

Professor Shamos published this paper in April of 2004. This paper is
deeply flawed, but it deserves a careful response for three main reasons:

1) The OVC is flatly opposed to invisible ballots (DREs) created with secret
software. Shamos argues in favor of invisible ballots.

2) Professor Shamos in one of few prominent scientists that argues in favor
of invisible ballots. His testimony is often used by organizations seeking
to bolster their support of DREs.

3) Although some of his arguments appear to be wrong, Professor Shamos makes
many excellent points worthy of consideration and support.

Shamos shows a fondness for defeating arguments no one is making. The paper
is full of strawmen. It appears that Shamos has not really followed or
considered what some of the leading thinkers in this area have been saying.
He mischaracterizes or ignores them.

The title itself is weak. "Paper v. Electronic Voting Records" is not really
the issue here. We want to know where the authentic vote exists. Should it
be purely electronic? He does not consider the possibility of paper ballots
(where the actual vote exists) produced with computerized voting systems
where there is also an electronic audit trail. He does not discuss ideas
for reconciling paper and electronic records.

He starts by listing eight claims made by DRE opponents. Then he says,
"Each of these arguments will be examined in this paper and found fatally
flawed.." Could it be that he constructed these 8 arguments in such a way
that they could be easily refuted?

     1) Voting machines are "black boxes" whose workings
           are opaque to the public and whose feedback to the
           voter is generated by the black boxes themselves.
           Therefore, whether or not they are operating properly
           cannot be independently verified and the machines
           should not be used.

The issue here is not so much about whether they can be independently
verified: it's that they aren't independently verified [to be operating
properly]. Certainly, they cannot be verified with black box testing alone.

     2) No amount of code auditing can ever detect malicious
         or even innocently erroneous software. Therefore the
         machines should not be used.

Again, this is not really the issue. It's not about whether the auditor can
spot malicious or erroneous code during a code audit, it's about whether or
not they will. Given the track record of code that has been certified, it
appears auditors have a very limited focus in these code audits.

     3) No feasible test plan can ever exercise every possible
         combination of inputs to the machine or exercise every
         one of its logic paths. Therefore the machines should
         not be used.

I suppose that every professional test engineer knows that the first
sentence is absolutely true [for software of medium or better complexity].
This fact by itself is not why paperless voting systems should not be used,
but it's part of the reason.

     4) Hackers can break into the FBI's servers and deface its website.
         It ought to be child's play for them to throw an election.
         Therefore the machines should not be used.

Who is making this argument? Generally, it's true that hackers come up with
remarkable tricks that no one thought possible.

     5) DRE machines have been plagued by a host of failures all
         around the country. Therefore the machines should not be used.

These failures illustrate some of the costs/benefits of DREs. Right now, it
appears that many jurisdictions have spent a lot of money on technology that
is immature and that will be obsolete soon. It just looks like a bad
investment.

     6) The DRE industry is dominated by a small number
         of companies, some of whose executives are announced
         supporters of the Republican party. An executive could
         command his programmers to add code to each machine
         manufactured by that company to move votes to a
         favored candidate, thus determining the outcome of the
         election. Therefore the machines should not be used.

While some have characterized paperless voting as a Republican conspiracy,
this is small minority of critics. Interestingly, we are seeing some
Republicans saying that Democrats will use paperless systems to rig the
vote. The OVC position is that it must be assumed that all people involved
in election administration, as well as all the voters, are partisan. The
integrity of the voting system must not depend, at any point, on people (or
groups of people) being honest, non-partisan, or uninterested in the
outcome. The integrity of the voting system can only be assured with a
system of checks and cross-checks.

     7) Many prominent computer scientists have said that DRE
         machines cannot be trusted. Therefore they should not
         be used.

It's not so much that so many have said that. It's what they say about it.

     8) If added to a DRE machine, a voter-verified paper trail
         allows the voter to satisfy herself that her voting preferences
         have been recognized correctly by the machine. Therefore,
         the voter-verified paper trail solves every one of the
         aforementioned problems and every DRE machine should
         be required to have one.

No one is making this argument. This is a pure strawman.

Shamos rambles on saying, "Since the Industrial Revolution, man has chosen
to rely on machines for tasks.." This part has some interesting points, but
none of it has anything to do with paperless voting. We all know that
technologies bring various risks as well as advantages. Shamos completely
misses the point.

The point with DREs is the possibility of rigged elections with no
possibility of recovering how voters actually voted. We are suspicious of
malicious insiders, and for good reason. If conspirators are given a way to
throw an election, we must assume they will try since we know it has been
done in the past. Cheaters are everywhere.

If there is a large enough conspiracy, no amount of careful voting system
design can prevent it. However, we can make a conspiracy unlikely by
requiring such a large amount of cooperation that it is bound to fail. The
weakest voting system would be one where a single conspirator could throw an
election. Paperless voting introduces the possibility that a single person
with the requisite knowledge and access could throw an election. We can
also imagine scenarios with a few insiders with a few outside confederates
that could change the outcome of an election.

Shamos argues that we can make aircraft software reliable, so we can trust
software for voting machines without the need for a paper audit trail. His
analogy does not hold. The threat model is not similar. Safety in aircraft
software is a goal common to all involved. Everyone wants it to be safe to
fly. Shamos mentions that planes have been deliberately crashed but this is
extremely rare.

We find substantial agreement in Section 3.2 regarding open source. Shamos
concludes, "On the other hand, there is no reason that the ballot setup,
display, tabulation and reporting sections of voting system code should be
kept secret, and manufacturers would be wise to accede to public demand in
this regard."

Section 3.3 has some good suggestions for handling DREs, but doesn't this
also show some of the hidden costs of DREs? More time, expertise, manpower,
etc. are needed to ensure the integrity of these machines.

I fully agree with Section 3.4. Probably, Shamos was not talking about the
OVC as coordinator of this federally funded effort to develop standards.
But I think we are developing a very strong group of scientists and
engineers that could do this work.

In section 3.5, Shamos talks about some parallel testing that was employed
with DREs. He admits that it has limited value. But it's worse than that.
This type of testing is very expensive since it requires another DRE for
each pollsite, and can only find certain types of problems that are unlikely
to occur. He says, "It is designed to detect the nightmare scenario in which
some agent has tampered with every machine in the jurisdiction undetectably,
a major risk cited by DRE opponents to justify the addition of paper
trails." It really has almost no value the way it is described because it
wouldn't even detect what he says it is designed to detect. That is, it's
possible that every machine has been tampered with while parallel testing
would not detect it because the tester does not know the trigger for putting
the machine in rigged mode.

Shamos is at his absolute worst when he says, in effect, to the people that
say these machines could be rigged (or have been rigged), "show me." He
wants people to show him how this has been done or could be done. It
apparently means something to him if no one shows him.

Why would anyone be willing to show him? Consider the case of slot machine
rigger, Ronald Harris. Suppose you were defending the slot machines for
their lack of bias. Would it be particularly meaningful to issue a
challenge to see if anyone could hack one of these machines? Before he was
caught, would Ronald Harris have been interested in meeting your challenge?
Even if you offered a reward of $10,000 or more, why would Harris be
interested in revealing his scheme when he could reap hundreds of thousands
or even millions by keeping his secret? In fact, Harris was a slot machine
examiner that figured out a way to insert code such that the machine would
payout the jackpot if you inserted coins in a certain pattern. If you know
the combination ("signal string"), you get the jackpot: Otherwise, it
behaves just like every other slot machine (Harris was only caught because
his confederate acted very suspiciously after winning a $100,000 jackpot,
and Harris was found in the confederate's hotel room).

Now consider the stakes involved in just local elections. Billion dollar
projects have been won or lost with a single vote in the City Council.
Local officials are often involved in decisions that involve many millions
of dollars. If someone has figured out a scheme for rigging voting
machines, they will not be interested in telling you about it for the same
reason Harris would not have been interested in telling you about his slot
machine rigging scheme. If they have successfully tested the scheme in an
election, they would be guilty of a felony and probably will not want to
admit that. Furthermore, if they took such a risk, they probably are
expecting some large future rewards. They may be hoping to make millions by
throwing a single local election. They won't be interested in telling
anyone about it in advance (other than co-conspirators).

After strenuously arguing that it couldn't be done, Shamos seems to admit
that it could be done. But he dismisses the threat because it would only be
local.

"It is possible that in a conspiracy a tamperer's confederate could, while
voting, provide information via touchscreen selections or the write-in panel
that could inform the software of the particular voting positions to
manipulate. However such an act would have local effect only, since it
would take one confederate for each voting machine involved. It would not
be feasible to perform manipulation on a large scale with such a scheme."

Is Shamos trying to say that unless you can overcome a several percent
difference nationwide in a presidential contest that it's not important? I
don't think we can dismiss "local effects." As previously mentioned, local
contests (City Council, County Supervisor, ballot measures, etc) can carry
very large financial impacts. And local effects could even decide a
national contest in a Florida 2000 situation where a few hundred votes swung
one way or the other could make the difference. The voting system is as bad
as its weakest link. Even if a particular type of manipulation cannot be
done on a large scale, it is unacceptable to permit it.

4. Answering the Objections
-----------------------------
Shamos goes over each of the eight objections he identified at the outset
and attempts to summarize how he has defeated these objections. Some of
these summaries are truly incredible. For objection no. 7 (computer
scientists say DREs are bad), he uses his estimate that "About 100 of them
have signed a resolution in favor of paper trails proposed by
www.verifiedvoting.org" to conclude that "the other 9,999 out of 10,000 have
remained open-minded on the subject." His math here is positively shameful.
The ACM poll is currently running 95% against DREs (in favor of voter
verified paper trails). The list of independent (i.e., those not on the
payroll of DRE makers) computer scientists speaking out in favor of DREs
seems to begin and end with Shamos.

Finally, Shamos cites voter disenfranchisement due to poor absentee ballot
systems. He says, "If computer scientists are truly concerned about threats
to democracy, that's one they should work on." He has mischaracterized this
as an either/or option. This is not a choice we have to make. This is
just another big problem-one of many-with the voting system. If we want to
have a great voting system instead of the bad one that we have now, there is
a lot of work to do. It's a very big job.

Alan D.
_______________________________________________
OVC discuss mailing lists
Send requests to subscribe or unsubscribe to arthur@openvotingconsortium.org
==================================================================
= The content of this message, with the exception of any external
= quotations under fair use, are released to the Public Domain
==================================================================
Received on Sat Apr 30 23:17:05 2005

This archive was generated by hypermail 2.1.8 : Sat Apr 30 2005 - 23:17:22 CDT