This is what secure tabulation looks like

Up to now, the Wisconsin Elections Commission’s interest in elections security has focused on the voter-registration system (WisVote), rather than the vote-tabulation system (the voting machines). When the Commission has paid attention to concerns about voting-machine security, it typically has been for only as long as it took commissioners to ask the vendors “Tell us how to refute these concerns.”

The Commission has also made a habit of limiting its own information sources. Earlier this year when they felt the need for advice on election security, they convened an Election Security Advisory Panel consisting entirely (I swear I’m not making this up) of county and municipal clerks. That was a revealing indication of the Commission’s level of interest in seeking advice from anyone else … say, disinterested IT professionals or highly interested, well-informed voters.

This image has an empty alt attribute; its file name is CAPQuote.jpg

But the Commission’s interest in voting-machine system security may be showing signs of life.

Last week, the Commission announced the formation of a new Elections Security Council of “federal, state and local partners” that will “formalize collaboration between these key groups and the public to improve communication and maximize election security.”

As usual, the Commission’s idea of “key groups” is limited to government officials. It’s possible their idea of ‘communication’ remains limited to outgoing messages to reassure voters that all is well.

Oh, well, it’s a start. Give the new council a chance to join the fight for voting-machine security. We’ll know more after their first meeting on October 16, when they will discuss whether and how they want to involve any other stakeholders.

Realistically, though, it’s possible this new council will — as the Commission itself has always done — focus its efforts exclusively on the voter registration system (WisVote) rather than voting-machine system security. Nothing in the press release specifically indicated the Commission is looking to expand its election-security efforts beyond WisVote.

Nevertheless, just in case this council represents an awakening, its members should know what a secure tabulation system would look like.

So here’s a gift to the new Elections Security Council:
A list of what would be in place if our voting-machine system was secure.

Most of the elements listed below are common sense, not rocket science. It’s just sensible, prudent management of a highly critical IT system. Some elements are present for Wisconsin. Others are missing. State and local election officials cannot create all the missing elements, which means they need to look for ways to make up for their absence.

If any members of the new council are curious to know which of these elements are in place and which are missing, multiple nationallyrespected electionsecurity authorities stand ready to share critical insights. Those experts’ interest in security is unaffected by financial interests and by any reflexive defense of the status quo.

In a secure vote-tabulation system:

Voting equipment manufacturers would…

  • Manufacture only those systems that are as secure as possible given current technology and customers’ budgets.
  • Manufacture only systems that use or produce ballots that voters have verified as accurate records of their intent, and that allow local officials to verify the votes were tabulated accurately.
  • Cooperate fully with the federal Department of Homeland Security monitoring of the companies’ own computers and security practices.
  • Cooperate fully with state and local governments’ security requirements.

The federal government would…

  • Promulgate strong, clear, and frequently updated regulations for secure, auditable voting systems, and for the independence of private testing labs.
  • Actively and rigorously apply those regulations when certifying new systems or updates.
  • Actively monitor and enforce compliance with those regulations.

The state government would…

  • Through law and regulations, implement strong security and auditability requirements for voting systems used in this state, and rigorously enforce those through certification.
  • Provide guidance and technical assistance to local governments related to voting-machine system security, so that vendors are not their customers’ only source of information and advice.
  • Adopt laws and regulations for local governments’ voting-system security practices.
  • Monitor local compliance with required voting-system security practices, and have the ability to correct poor practices.
  • Coordinate strong post-election tabulation audits, involving all the counties’ boards of canvassers, that verify the correct winners in all statewide races before certification.

County government election officials would…

  • Follow federal and state requirements for securing county elections-management system hardware and software.
  • Have professional IT staff capable of and assigned to working with the voting-system vendor on security-related matters. (If not county staff, an independent contractor who is unaffiliated with voting-machine sales and service.)
  • On Election Night, obtain electronic election records (including CVR and digital ballot images) from municipalities. Maintain strong internal control and to support voter confidence and ballot security, post digital ballot images to the internet within 24 hours of poll closing.
  • During the county canvass, use the paper ballots to verify that the computers identified the correct winners. If problems are found, correct results before certification.
  • Between elections, audit various election-security practices and take action to improve whenever any issues are found.

Municipal government election officials would…

  • Maintain year-round strong internal control of marked and unmarked ballots; other election records (e.g., CVR, digital ballot images); and voting-system hardware and software.
  • Maintain equipment according to manufacturer recommendations. Routinely and reliably inspect equipment inside and out for signs of tampering or malfunction; take action to correct any issues noted.
  • Conduct strong pre-election testing of both tabulators and ballot-marking devices; take action to correct any problems noted. Make sure all voting machines are equally reliable and operable.
  • Train election workers in how to maintain security; how to notice trouble signs; how to document and respond to trouble signs or lapses.
  • Monitor performance of elections workers to ensure that no bad habits develop, that any departures from standard procedures are quickly noted and corrected.

Voters would…

  • Volunteer to serve as poll workers and hand-counters for audits.
  • Pay attention to election security issues, getting neither too excitable nor too complacent.
  • Be willing to hold their local officials accountable for verified accurate election results.

WEC to voters: Voting machines use binary code, so you don’t need to be able to decipher your ballot.

No, you’re not crazy. It doesn’t make any sense.

Today the Wisconsin Elections Commission once again took up a voting-machine vendors’ request to market a new product here. Once again, the Commission confined voters to five-minute comments and then invited voting-machine vendors to sit down at the table with them to pitch their products.

Once again, the Commission discussed the voters’ concerns only for the purpose of asking the vendor to refute them.

And then the Commission once again approved a ballot-marking device (BMD) that records our votes as barcodes we cannot read.

The machine in question today is called the ExpressVote. Designed primarily for voters who cannot use a pen, BMDs require voters to use a touchscreen to indicate their votes. The computer then prints a marked paper ballot. Increasingly, BMDs are being promoted to voters without disabilities, particularly early voters.

Some BMDs print ballots that are nearly indistinguishable from hand-marked paper ballots, so that ballots cast by voters with disabilities look just like everyone else’s. Both voters and tabulators look at the same input to read the votes.

Bad BMDs, like the ExpressVote, print ballots that look like large cash-register receipts. On these ballots, votes are recorded as barcodes. This prevents voters from verifying that their votes were printed correctly. In addition, these machines violate voters’ privacy when a polling place has only one or two voters with disabilities. (More about barcoding BMDs here.)

But why?

You might ask (as most people do) why anyone would build such a feature into a machine.

You might ask, but the Wisconsin Elections Commission doesn’t.

Commissioners never asked the vendor: “Why? Why are you offering us a machine with this weird feature, when we know you can manufacture machines that perform all the desirable functions and none of the dicey ones?”

Whatever the answer is, it must not make the barcoding BMDs look good.

The vendor’s defense attorney

At one point, Chair Dean Knudson sympathetically acknowledged that voters who use barcoding BMDs can independently verify their votes only if they bring a barcode reader to the polls. He wisely noted that’s too much to expect of voters.

But beyond that, the commissioners’ questions could all be paraphrased: “How can we refute the voters’ stupid concerns?”

“Motivated reasoning” is the chop-logic that appears when people pick a conclusion first and go looking for reasons to justify it afterwards. For example, commissioners and staff repeatedly reminded each other: “We saw no problems when the barcodes were tested/audited/recounted. Therefore, we conclude the system is safe.”

That’s a textbook case of motivated reasoning. People whose brains are engaged know that hackers don’t avoid a system simply because it worked well during the manufacturer’s demo or the customer’s test, that computers do not earn magical immunity from future problems by working well on a previous occasion.

They know it’s a bad idea to give every questionable voting system one freebie botched election before rejecting it.

Commissioner Mark Thomsen, in particular, took it upon himself to play defense attorney for the vendor. He acted insulted that voters had implied the barcoding BMDs are “hackable.” But that wasn’t the voters’ point. Of course the barcoding BMDs are hackable; all computers are. If Thomsen had been listening to understand rather than listening to refute, he would have understood the issue was not “hack-ability,” but that barcodes remove voters’ and officials’ ability to detect hacking.

Most bizarrely, Thomsen repeatedly reiterated one laughable argument made by the vendor. The argument is this: Because the tabulators read all votes as binary code, voters have no reason to object when the printer makes their votes indecipherable to humans.

Thomsen has more than enough intellect to understand that users need to be careful to feed computers only accurate information, so he understands why voters need to be able to tell whether their intended votes were correctly recorded on their paper ballots.

The voter registration system, like the voting machines, processes information as binary code, but I have no doubt Thomsen would immediately see the problem if anyone suggested that WisVote render each voter’s registration record unreadable to the voter.

But for some reason he pretended he didn’t understand.

Thomsen even went on to argue in favor of another type of BMD that WEC staff had wisely recommended rejecting. This machine combines a ballot-printer and a tabulator in one machine, creating a feature that independent elections-technology experts ridicule as the “permission to cheat” feature. Fortunately, the other commissioners acted as a wise jury, so the notion of overriding the staff recommendation to reject that component went nowhere.

Voters shouldn’t give up.

The commissioners are neither stupid nor crooked, as far as I can tell. For example, when they’re working on security for WisVote (the voter-registration system), they do a fabulous job.

It’s only when the questions involve the tabulation system that they devote their energy to making excuses for security flaws rather than fixing them. They suspend their common sense only when the voting-machine vendors sweet-talk them. But whatever the reason, siding with the voting-machine vendors against the voters is something of a habit.

As voters who want to protect our own votes and our communities’ elections, we’ve got work to do. We need to show up and object every time the Commission considers idiotic equipment. I see too much common sense on that commission to believe they will keep these particular blinders on forever.

The other suggestion on the table is a lawsuit. Wisconsin law requires that voting systems “permit an elector to privately verify the votes selected by the elector before casting his or her ballot.” If the WEC admits the barcodes are the only marks ever counted as votes, they will be admitting that the BMDs don’t comply with the verifiability requirement. On the other hand, if the WEC argues that the voters can verify the human-readable text on each barcoded ballot, they will be stuck with no explanation of why that text is never counted as votes. Therefore, if we can find a lawyer willing to defend election security and voters, we could make an argument that barcoding BMDs are already illegal in Wisconsin. If the Commission wants to build voter confidence and enhance security, it will adopt this line of reasoning even without a lawsuit.

About those Russians…

In the past two weeks, three reporters have asked me to comment on Russian interference in US elections. Do I believe the Russians interfered with the 2016 election? Do I think they will try in 2020? And my least favorite: Do I think Russians are the worst threat to the voting machines?

I’ll answer the ‘worst’ question first: What the hell does it matter?  All threats are threats. Will it be a boring news story if our election is stolen by a Canadian anarchist living in his grandmother’s basement, or by a random computer glitch?

I’ll tell you what the worst threat is. It’s the threat that is literally the sum total of all other threats. Wisconsin county clerks are STILL not using the only safeguard effective against every voting-machine threat including the Russians: Using our paper ballots in prompt, routine, hand-counted audits that verify the correct winners.

The simple truth should be obvious. It is ridiculous to allow any computers to make any big decision unless you have a reliable way to detect and correct serious computer errors.  

Can you think of any other government agency that relies on computers and doesn’t have some way to notice if the computer screws up a big operation? No, you cannot. There isn’t one. Only election officials trust their computers that blindly, and demand our trust, too.

When Wisconsin’s county clerks declare election results final without verifying the correct winners, they are allowing computer programmers to pick the candidates who will govern us.1 They don’t supervise these programmers. They don’t know even know who or where they are.2

As to the other questions:  I don’t know whether the Russians or anyone else tampered with the voting machines in 2016 and 2018. No one does because no one checks.3 How is that not scandal enough?

Wisconsin’s election officials just seal our paper ballots on Election Night and leave them sealed until it’s time to destroy them two years later. No one ever knows if the paper ballots tell a different story than the computer tapes.

And I don’t know whether Russian criminals are planning to mess with the voting machines in 2020. I do know it is wise to assume they are. Most importantly, I know it will be criminally negligent if our county clerks make no effort to detect and correct any hacks that might get by the security system.

Call your Wisconsin County Clerk today and say: “Surely you understand that you cannot guarantee the security of our voting machines. Too much is outside your control. The only thing you can secure is the election results, and you can do that only by using our paper ballots in hand-counted audits during the county canvass to make sure you certify only the correct winners. Get busy now on developing audit procedures for the 2020 elections.”

– – –

1 A few Wisconsin county officials claim they “program their own voting machines” and imply that provides security. They don’t, and it doesn’t.
The county clerks ‘program’ the machines only in the sense that you ‘program’ a new cell phone with your personal address book and settings. If any are messing with the actual tabulation software, they are breaking federal law. Truth is, these county officials rely on the voting-machine company in the same way you rely on Samsung, Apple, or Nokia.

2 Example: In 2016, election-security advocates noticed that Dominion—the nation’s second-largest voting machine company, which counts many Wisconsin votes—was recruiting programmers in Serbia. The company’s official response was: “Like many of America’s largest technology companies, which develop some of the software for their products in places like Asia, India, Ireland and the Mideast, some of our software development is undertaken outside the U.S. and Canada, specifically, in Serbia, where we have conducted operations for 10 years.”

3 In the 2016 recount, half of Wisconsin’s presidential votes were “recounted” only by running the ballots back through voting machines programmed by the same people who programmed them for Election Day. These were the ballots in the state’s largest counties (except Dane)–the counties most at risk of hacking.
In the half that was hand-recounted, the recount found that more than 1 in every 170 votes had originally been miscounted. These errors were not deliberate and affected both major-party candidates equally. As a result, they did not change the outcome and the news media didn’t report it.
But notice this: even when that many votes had been miscountedup to 30% in some individual wardscounty clerks did not notice it in their regular canvass. They detected the incorrect vote totals only when forced to check their work with a recount. Unless our county clerks adopt routine audits, the same will happen when hackers put the Election-Night results outside Wisconsin’s microscopic recount threshold (0.25%). There won’t be a recount and the hackers will have successfully pulled off their crime.

What will happen when election hackers get to Wisconsin? A black comedy

Last week, a reporter and I were discussing election hacks that might happen in Wisconsin. He has done his research and understands the threats. He posed an interesting question: What if hackers wanted only to create chaos and distrust, rather than change the outcome of a statewide election?

Hmmm…what would happen? I thought through the likely chain of events and realized it is not possible to create distrust by hacking a Wisconsin election — but not for the reason you would hope.

If this was a movie, it would be a black comedy with a twist ending. The big gasp would come when the election thieves (along with the viewers) realize the fatal flaw in their plan …

Scene 1 opens in the messy office of computer hackers. They are working for a foreign government that has its eyes on the US presidential election. They are celebrating because they just succeeded in compromising a small voting-machine service company in eastern Minnesota.   

Scene 2 takes place in the Intelligence Headquarters of the foreign capital. The hackers are reporting their progress to the chief.

“The good news,” they say, “is that we know how to deliver compromised software to all that company’s customers. Local election clerks will never know. They never inspect the software and their cute pre-election tests cannot detect hacks that activate only on Election Day.”

“The bad news is that the company controls only some of Wisconsin’s voting machines. They don’t have enough votes to deliver a statewide race.”

The intelligence chief doesn’t care.

“No worries,” he says. “If we show we can hack the machines, we will destroy trust in the process even if we don’t control the outcome. Just cause chaos, sow doubt. Whoever wins won’t have legitimacy.”

“Go for it,” he says. “Pick whoever you want to win in the machines you control. Just as long as the results are suspicious.”

Scene 3 is back in the hackers’ office. The hackers are gleefully developing their plan.

As voters cast their ballots, the hackers will let the voting machines count their votes correctly.

But on Election Night, when the poll workers push “tabulate,” the computer will quickly flip the vote totals of the top two candidates in each primary. The voting machines will give the biggest vote total to the second-place finisher, and make the voters’ choice come in second. Not a single polling place in the entire area will report accurate results. 

Scene 4 takes place on Election Night, April 7, 2020. Poll workers are gathered around a voting machine in a small city in western Wisconsin. The chief inspector pushes a button on the back as others eagerly watch the poll tape emerge. Expressions of surprise.

Cut to the Associated Press Election-Night newsroom. Much excitement. An editor shouts to a reporter: “Go figure out what’s up with Wisconsin’s rural voters! That’s not what anyone predicted they would do, in either party primary.”

Scene 5 consists of a montage of cable-news soundbites on Wednesday, April 8, 2020.

Questions abound:  “What’s going on in rural Wisconsin? Why did the voters in both major-party primaries confound expectations?”  

The talking heads burble on: Maybe voters lied to the pollsters about who they would vote for, or whether they would vote at all. Maybe hostile cross-over voting went both ways…maybe the leading candidates were too confident…

In Scene 6, viewers get the shocking revelation.

It’s now two weeks after the primary. A county clerk and two senior citizens sit in a drab conference room in a small county courthouse. They are finishing up the official canvass. The clerk says: “I printed out the certification statement. This is one election we won’t forget.”

They pass the paper around. Each one signs it. The hacked results are now official.

One of the board of canvass members remarks: “I’ve been doing elections work for 35 years, and voters still surprise me. Well, let’s go for a beer.”

Cut to the Intelligence HQ in the foreign capital. The Chief is furious; the hackers stare at their shoes. 

The Chief slams his fist on the table: “You idiots chose a state where no one would even notice an election hack!!! Why didn’t you do this in Colorado or New Mexico?

“How did you expect your hack to be noticed when Wisconsin’s paper ballots are sealed up on Election Night and never seen again?

“Didn’t you know that Wisconsin election officials never audit the primary elections?

“Didn’t you bother to notice that Wisconsin never recounts unless results are virtually tied?

“You bozos! Get out of my sight!”

The hackers leave. The chief smiles and picks up the phone.

“Mr. Secretary, good news. We just confirmed the people in Wisconsin trust their computers implicitly. No one will notice manipulated results — not officials, not reporters, no one. Senate, Governor, President, pick whomever you want. ”

He hangs up the phone and calls his assistant in. “Contact the team who is working inside that big company, ES&S. Tell them to add Wisconsin to the list for November.”

* * *

Wisconsin’s local election officials do not stand a chance against sophisticated international cybercriminals. Too much is outside their control. Too many vulnerabilities, known and unknown, threaten the tabulators. Securing Election-Night results is a wishful fantasy.

But Election-Night results are preliminary and unofficial. Final results are the ones that matter and that could be secured — relatively easily, too. County clerks could use the paper ballots and their administrative authority to order hand counts. Simple audits could verify the winners while the clerks still have time to correct any miscounts.

But Wisconsin county clerks refuse to do that, so our elections continue largely on an honor system.

The Wisconsin Elections Commission orders scattered audits of individual voting machines after November elections. That’s grounds for some hope. But even with improvements made in 2018, if these audits ever detect a miscount, they are as likely to cause chaos as to prevent it. They are not rigorous enough to verify the correct winners and are not binding on final results. Officials have no agreed-upon procedures for what they will do if auditors detect that the Election-Night results were miscounted.

Contact your Wisconsin County Clerk. Tell him or her to develop written canvass procedures — NOW — to verify the correct winners in the 2020 elections before they certify the final results.

Voters ask for security. The Wisconsin Elections Commission gives only reassurances.

Summary: The WEC are pressuring voters to accept insecure election equipment, when they should be pressuring vendors to improve it. 7-minute read.

* * *

June 27, 2019 – Imagine two friends walking down the street when a masher starts to hit on the woman—even tries to get her into his car.   

The woman’s friend doesn’t need to throw any punches. But he should at least say: “Hey, buddy. You’re out of line. Move on.”

The last thing you expect him to do is to tell the woman: “You need to trust. This guy’s offer sounds legit. He’s got a nice car. Go ahead; get in.”

That’s like what’s happening with WEC (the should-be friend); voters; and voting-machine companies (the masher). Voting machine companies are offering risky equipment, and WEC is pressuring voters to get in the car:

  • Some models of ballot-marking devices (BMDs) print ballots with the votes recorded in barcodes, rather than in marked ovals beside candidates’ names. BMDs are necessary for people who cannot mark their own ballots. The problem is that barcoded votes make it impossible for voters to verify which candidates will get their votes. Even a voter carrying a barcode reader wouldn’t be able to tell whether 02060101 was the right candidate. The barcoded ballots also print the candidate’s names as text, but the computers count only the barcoded votes.  The Commission has approved two barcoding BMDs:  the ES&S ExpressVote and the Dominion ICE.
  • Hybrid voting machines combine a ballot-marking device and tabulator in one machine, which sounds okay until you know that, after a voter has inspected the ballot and inserted it back into the machine for counting, the machine passes the ballot back under the printer head. As a result, the machines can be mis-programmed to print additional votes on the ballots or to make marks that invalidate the ones the voters made.  Like barcoding, that makes voter verification impossible. Dominion ICE is the only hybrid voting machine currently in use in Wisconsin.

 

Manufacturer ES&S recently asked the Commission for permission to sell an updated voting system that includes both a safe BMD (the Automark) and a risky, barcoding BMD (the ExpressVote).  The Commission took the matter up at their June meeting.

In advance of that meeting, dozens of voters contacted the Commission, asking them to deny approval to the barcoding machine. After reading the voters’ emails, the commissioners saw a problem, but it wasn’t the security flaw. The problem they saw was voter resistance to the security flaw.  (With one exception–see the footnote)

When they met on June 11, neither staff nor commissioners were coy about the purpose of the meeting. Administrator Meagan Wolfe introduced the staff who “conducted the campaign to approve the voting equipment.” (Staff are campaigning to approve the equipment? Shouldn’t they at least be impartial?) (At 11:30 in this video recording of the meeting.)

Commissioner Mark Thomsen was equally clear about what he wanted from the meeting:  “I’d like to be reassured about any security issues and that the public knows that we don’t have a problem there.”  (38:38 in that video) (Shouldn’t he instead like to be educated about the security issues? And want the public to be well-informed?)

Were these mere figures of speech? Did Wolfe and Thomsen instead mean to say that they wanted to conduct a rigorous assessment to reduce or eliminate the risks?

No, they did not. Watch that video and you will see staff, commissioners, and ES&S sales representatives working together with shared and very limited purpose: To convince each other and the voters that all is well.

Had commissioners come to the meeting ready to grapple with and resolve the security issues, they would quickly have posed the obvious first question, given the controversy: Why are barcodes used at all? Particularly when it is demonstrably possible to manufacture a machine with all the desirable features of the ExpressVote but without the barcodes? What benefit do the barcodes provide, to whom, that justifies degrading voter verifiability like this?

But no one asked. So no one answered.

That wasn’t the only important question unasked and unanswered. Tony Bridges, the Commission’s Election Security lead, reassured the commissioners that the votes recorded as text, rather than the barcodes, will be counted in recounts and audits.  (Starts at 48:12 in the video linked above.)

The Commissioners know — even if Bridges does not — that in 2016 the Commission testified in court that statutes give counties, not the Commission, authority to decide whether to recount by hand or by machine. The judge agreed. So Commissioners know they cannot require the recount method Bridges described. Yet no one corrected him. The stated purpose of the meeting was to increase confidence in election security — and Bridges’ misstatement did that.

Several commissioners are lawyers. If they had been engaged in assessing risks rather than excusing them, they surely would have also noticed that no Wisconsin statute anticipates that votes will be recorded twice on the same ballot. That creates a rat’s nest of legal questions around barcoded ballots: Which is the ‘real’ vote?

What does it say about the validity of results in un-recounted races when the Commission insists, as Bridges suggests, that only those votes recorded as text are reliable enough to decide a recount? One of the candidates in the next contested recount might suspect he or she got more votes from the barcoded ballots. When that candidate challenges the hand-counting counties, what legal argument will the Commission suggest to those clerks to defend Bridges’ method against pressure to recount by machine?

Bridges’ proposal is just as problematic for voting-machine audits. For years, the Commission has repeatedly asserted that this state’s audit law, s.7.08(6), Wis. Stats., requires auditors to read the votes the way the machines are designed to read them. Reading only the text votes from barcoded ballots cannot fulfill that requirement, because the tabulators don’t use those votes. So Bridges’ proposed audits do not qualify as s.7.08(6) audits. Yet those are the only audits the Commission has authority to order.

None of that came out in the meeting, however, because the commissioners were wholly fixated on defending the barcoding BMD. Having built up the illusion that officials will routinely check the barcodes for accuracy, Bridges’ testimony was on script.

The manufacturer’s claims were also accepted without question. None, however, stand up to even simple critical examination.

  1. The manufacturer’s first argument is: “The voters can verify the votes that are printed as text.”
    If any commissioners had been working to protect voters, they would have said: “We cannot consider something that’s never counted to be a ‘vote.’ So we don’t see the value in verifying the text. It’s the votes that will be counted that must be verifiable to comply with the law that says that voters must be able to privately verify the votes selected.”  
    No commissioner said that.
  2. The next argument is: “Audits and recounts will notice if the barcoded votes differ from the printed text.”
    If any commissioners had been working to protect voters, they would have told the manufacturer that is irrelevant in Wisconsin. Here, very few races are protected by recounts because recounts are allowed only when preliminary results are too close to have been hacked (Manipulated results will surely have a victory margin larger than 0.25%.) Audits protect even fewer races because the Commission has no authority to correct election results even if an audit detects a problem in the sampled machines.  
    No commissioner brought that up.
  3. Manufacturers offer a third defense when they are forced to admit the votes printed as text are merely decorative. They have built a feature into the BMDs that allows a curious voter to reinsert the ballot into the machine that printed it, which will read the barcode and display the votes on a monitor.
    If any commissioners had been working to protect voters, they would have pointed out that’s not verification. It requires a voter to trust the machine once to print the correct votes, and then trust it again to read back the correct votes. If a barcoding BMD is programmed to print the wrong votes in the barcode, it will also be programmed to read the right votes back to the voter.
    No commissioner pointed that out.
  4. For their final line of defense, ES&S falls back on obfuscation. The manufacturer explains that the tabulator uses the same set of codes to interpret both marked ovals and barcodes. For example, the code assigned to a candidate whose oval is located in the second row of the sixth column on the first side of the first page of a printed ballot would be 02060101. The barcoded votes for that candidate contain a reference to that same spot—02060101.
    If any commissioners had been working to protect voters, they would scoffed and said, “If the voter hasn’t verified it, we don’t care how the tabulator reads it. Stop yammering about irrelevant technicalities and bring us a BMD with the good features of the ExpressVote and voter-verifiable ballots.”
    None of the commissioners scoffed.

Staff contributed additional weak arguments to help ES&S sell their machines: 

  • Staff pointed out that they found no problems with the barcodes when they tested the systems. They did not mention that the machines used by Election-Day voters will be at risk of mis-programming, while the machines provided by ES&S for testing were not.
  • Staff said that previous recounts and audits found no problems with incorrect barcodes in past elections. They did not explain how that protects future elections.
  • Staff told commissioners that local officials often test the machines before each election. They did not explain that pre-election tests provide no security against malicious code, which would be designed never to reveal itself before Election Day.

No surprise: After this discussion, the Commission voted unanimously to approve the machine, and instructed its staff to reassure the voters.  A few days later, every voter who had urged caution got an email from Public Information Officer Reid Magney.  Following the commissioners’ instruction to convince voters that barcoded votes are “a perceived problem, not a real one,” (43:32), Magney uncritically repeated the manufacturer’s claims and even used the opportunity to distribute an ES&S marketing brochure.

So here’s a direct plea to the Wisconsin Elections Commission and their staff: Stop seeing it as your job to make the companies’ case to the voters. Start making the voters’ case to the companies.

When you hear manufacturers’ claims, make skepticism your default attitude. When you hear voters’ concerns, default to curiosity.

When a law or regulation can be interpreted either way, go with the common-sense interpretation that favors the voters’ interests. Don’t devote extra effort to wresting out an interpretation that favors the voting machine companies’ interests. (I’m looking at you, Staff Attorney Michael Haas—1:27:00.)   

Demand security from them, not trust from us.

In short, WEC, come over to the voters’ side where you belong.

Footnote: Chair Dean Knudson’s line of questioning, which starts around 50:00, was responsive to concerns about pre-election testing. However, to be fair, that line of questioning challenged only election officials to reduce the risks of barcodes. He did not challenge the manufacturer to eliminate them.

* * *

Note to the media:  Voters could use your help in getting the WEC to work on tabulation security, rather than to continue working on reassurance.
The security problems with barcoding and hybrid BMDs are being taken very seriously outside Wisconsin. Federal election-security legislation has been introduced that would prohibit their use. Senator Tammy Baldwin is co-sponsoring the Senate bill, SB 1472; Representative Mark Pocan voted for the House bill, HR 2722.
If you ever ask WEC about election security, be prepared to receive a list of measures they have taken to protect the voter registration system, WisVote. The list will not explain–unless you press–that those measures don’t protect the tabulation system.
The WEC might also list some things they have done related to securing the tabulation system. Before you file your story, notice which are guidance rather than binding requirements, and notice that none resolve the risks created when voters cannot verify their ballots, such as in barcoding BMDs or a hybrid voting system. 

If it takes a leap of faith…

The year was 1977, and my friend Gail was in the market for a cheap used car. One of the guys in our apartment building, Chuck, wanted to sell his Pinto. 

“No, Gail, no,” I told her. “Safety experts say the Pinto’s gas tank can explode in even low-speed rear-end crashes.  There’s talk of recall and lawsuits. If you buy this car, you will be in danger, and you won’t be able to resell it.”

Gail dismissed my concerns. “Ford wouldn’t be selling the car if it was a problem,” she said. “And besides, Chuck said the car has seat belts.” She thought for a second and couldn’t come up with any more ways to dismiss or minimize the risk. “I have faith it’ll be okay.”

My roommate backed me up. “Gail, I saw a Datsun B210 for sale on Johnson Street.  The B210 does everything the Pinto does, without the risk. Forget about the Pinto.”

The more we tried to reason with her, the sillier her arguments became.  She told us Chuck had done a good job cleaning his trash out of the car. She promised us she would minimize the risk by never filling the gas tank more than a quarter full.

She had turned off her brain when it came to hearing anything negative about the Pinto. Chuck seemed to have her under some sort of spell. 

Gail came to mind during the Wisconsin Elections Commission meeting last week.  The Commissioners were meeting to decide whether to approve an updated version of a risky piece of elections equipment, called the ExpressVote.  They were listening to the manufacturer, ES&S, as Gail had been listening to Chuck, with doe-eyed admiration, treating words of wisdom from anyone else like flies to be swatted away.  

The ExpressVote is a type of ballot-marking device (BMD). Voters use BMDs to mark their ballots when they cannot, or do not want to, use a pen.  BMDs don’t count votes; they just print out marked ballots. But (like any computer) they can be misprogrammed to print a ballot that contains different votes than the ones the voter intended.

Safe BMDs manage this risk by printing ballots that look just like regular hand-marked paper ballots.  Each vote is recorded once, as a marked oval beside some candidate’s name. The voter can see the mark is next to the correct name. The tabulator looks at that same marked oval, verified by the voter, when it counts the vote.

The Pintos among the BMDs—that is, the unsafe ones—print ballots on which the votes are encoded, in either QR or barcodes.  Encoded ballots record each vote twice—once in human-readable text, and once in computer-readable code.  Voters can verify only the votes printed in the text. The tabulator counts only the encoded votes.  If the BMD is programmed to print one vote in text and a different vote in the code, the voter cannot notice. The ExpressVote is one of these machines.

When a state election authority meets to approve voting equipment, they should invite the manufacturer, of course. But a truly rational, responsible commission would want all the reliable information they could get. So they would also invite independent experts to sit at the table to answer any questions that might arise and to comment on the manufacturers’ claims.

The mission of the responsible commission’s meeting would be to determine what is best for that state’s elections. The commissioners’ conduct—particularly their follow-up questions—would demonstrate that they wanted nothing less than complete, unbiased facts about the equipment.

But the June WEC meeting was not that.

We’d organized people to write to the Commission the week before their meeting, to explain the security risks and to pass along the independent security experts’ assessments. We explained the importance of voter verification and how encoded ballots prevent it. We told them of the gathering storm of litigation and prohibiting legislation. We explained that encoded ballots bring no benefit to balance the risk. We pointed out that other systems have all the same benefits and without the risks.

I reiterated that information in person, during the brief five minutes that the Commission allows for public comment at the beginning of each meeting. I asked the commissioners to protect our elections by turning away this pointless risk. They appeared to listen politely, but asked no questions. 

Then for the next two hours, everyone in the room was required to listen silently as the salespeople gave their pitch–no time limit for them! If the vendor stretches the truth, dissembles, or lies, the meeting rules provide the commissioners with no opportunity to obtain correction or rebuttal from an independent source. If the salespeople omit any important information, the rules of the meeting allow no opportunity for anyone else to provide it.

In short, the vendor, the commissioners, and staff all come to the meeting with a single, shared goal: to minimize or refute concerns about the security of the voting equipment and to approve it for sale in Wisconsin.

One example: The issue of voter verification.  ES&S designed a feature into the ExpressVote that allows a voter to reinsert the encoded ballot back into the machine, and have the BMD display the votes on a computer monitor for a second time.  ES&S wants everyone to believe that this feature provides voter verification.

But of course it doesn’t. Everyone in the room—commissioners, staff, and company reps included—was intelligent enough to know that if a hacker ever programs a BMD to print the wrong votes in the barcode, the hacker will also program it to display only the voters’ selections back to the voter. 

But the commissioners asked no skeptical or challenging follow-up question. None even bothered to wonder out loud why anyone would encode votes in the first place. A few even repeated ES&S’s claim of verifiability back to them, like my friend Gail, as if repeating illogic somehow makes it logical.

Another example: ES&S’s pitch regarding the safety of barcodes. Be forewarned: Don’t worry if you cannot see a connection between the following information and any security concern. There is none. The vendor claimed encoded ballots are safe because:

  • The programmer assigns each candidate a unique numeric code, based on that candidate’s location on the paper ballot. For example, the candidate whose oval is located in the 2nd column, 15th row, first side, first page of the ballot will be Candidate 021511.
  • When the tabulator looks at a hand-marked ballot and sees a marked oval at that position, the tabulator will count a vote for Candidate 021511.
  • When that same tabulator looks at a barcoded ballot and sees a barcode that translates into “021511,” the tabulator will count a vote for Candidate 021511.

Backed up with a glossy, illustrated, full-color brochure, the ES&S salespeople presented those facts as if they explained why hand-marked and encoded ballots are equally secure. But those facts answer a question that no one is asking and that no one cares about. The problem isn’t what the tabulator ‘thinks’ when it reads a barcode or a hand-marked oval.  The problem is that when given an encoded ballot, the tabulator reads and counts marks the voter cannot read or verify, unlike when the voter and tabulator look at the same marked oval.

Distracting the customer by talking about something else and pretending it addresses the concern is a time-honored marketing ploy. It works, too. Like gullible customers everywhere, the commissioners just smiled and nodded.

I wish I knew the causes of this smile-and-nod approach to approving voting equipment. I do know the commissioners are capable of being tigers when it comes to security of their own WisVote system, which handles our voter registrations. I’ve witnessed the commissioners asking intelligent, challenging follow-up questions—really engaging their critical faculties—when working through security issues involving WisVote.

Had our security concerns about something in the WisVote system, I’m confident the commissioners would have soberly instructed their staff to resolve the issue.  If we’d been talking about voters’ ability to verify their registration information, I cannot imagine any commissioner shrugging and telling me, “Well, it does take a leap of faith.” But that’s precisely how, during a break, one commissioner ended a conversation with me about voters’ inability to verify the votes printed on their ballots.

Yes, it certainly does take a leap of faith. And when it comes to security of our voting systems, that’s a very unwise way to do business.

Voting machine software delivered via internet? You betcha.

Summary: Dominion Voting, one of the nation’s largest voting-machine vendors, uses the internet to deliver voting-machine software to local election officials before each election. Local election clerks can be so naive that they will proudly say “The voting machines are never connected to the internet,” and genuinely believe that protects the software–even though they themselves downloaded the machines’ software from Dominion’s website onto a county computer, from which they made copies for each voting machine. 

* * *

Before we talk about Dominion in particular, a reminder about the basics: In Wisconsin (like everywhere else), every voting machine system (like every other computer) is hackable. Even if never connected to the internet, every working computer contains software copied from some other computer. And hacks don’t need to come in over the internet: Every computer is programmed by normally fallible humans who occasionally have motive, means, and opportunity for fraud.

That’s why every responsible manager, including every elections official, must routinely audit their computers’ output (that is, our election results).

Now, about Dominion Voting Systems and their Imagecast Evolution (ICE) machine.

Voters and reporters in the 12 Wisconsin counties* using ICE voting machines believe that their voting machines are never connected to the internet. What they probably don’t know is that (except for Fond du Lac County), their vote-counting software was downloaded from the internet anyway.

The software in our voting machines has to be updated for every election, because each election has a unique set of races and candidates. No election official in Wisconsin has the ability or authority to write these programs by themselves. They either send the information to an out-of-state vendor who will write the programs for them, or they use an app provided by their voting-machine vendor to compile the vote-counting instructions themselves.

Typically, when an outside vendor writes the software for voting machines, they will deliver it to the local election officials on portable media (something like a USB drive, an SD card, or a “PROM” pack) via courier or FedEx.

But Dominion Voting, a corporation headquartered in Toronto and Denver, emails the county clerk when the software is complete, and the county clerk then downloads the software from the Dominion website.

I first discovered this last year, when I was contacting the county clerks in an attempt to inventory their current security practices; get a read on their level of understanding of the risks; and assess their receptivity to the idea of protective election audits.

Here’s how it works: In Wisconsin, it’s the municipalities that own and operate the voting machines, but because the county clerk has overall responsibility for designing and printing the ballots and reporting the election results, municipalities in most counties cooperate to buy the same voting-machine system. They then rely on the county clerk to handle the machine preparation before each election.

The first four Dominion-ICE-using county clerks I interviewed were happy and proud to explain to me their pre-election procedures. When they get the email from Dominion before every election, they download the tabulation software to a county computer from the Dominion website; save it onto an SD card; copy it onto the county elections-management computer (which is never connected to the internet!), and from there copy it onto portable media to give to the municipal clerks to load into the individual voting machines.

As I spoke with them, I was trying hard to stay completely in a fact-gathering mode, to understand their point of view without influencing it. So I was trying hard to avoid asking follow-up questions like “Are you JOKING?!?!?).

But I do not have a poker face, and one clerk picked up on my discomfort. She patiently explained to me that it was safe to send voting-machine software over the internet because the Dominion website was secure and wouldn’t let her get to the software without a password. And because she downloads the software to a different computer–not the central county elections-management computer or any individual voting machine–she assured me that the local elections equipment stays “air-gapped” and secure.

I didn’t ask, but I imagined her thinking that any malicious code can be erased by waving the SD card through the air.

Yes, that is the level of IT sophistication typical of local election officials.

The fifth ICE user I spoke with was the atypical Lisa Freiberg, Fond du Lac County Clerk. Whew. 

Freiberg has enough IT sophistication and backbone that, when Dominion suggested to her that she rely on them to write the program updates and download the software via internet, she refused. Instead, she obtained from Dominion software that she maintains on the county elections-management computer. She uses that software before each election to design the ballots and write the instructions the voting machines will use to count the votes. When I interviewed her in July 2018, she believed she was the only ICE user in the state who refuses delivery of the voting-machine software via the internet.

ICE is not the only voting system that Dominion offers or supports, and I don’t know if the company sends any other system’s software out over the internet. But even without being an IT professional, I can see some of the opportunities this practice might offer to those who would like to manipulate our elections.

When even the New York Times cannot protect its email from hackers, we cannot expect the deputy clerk in a rural Wisconsin county to know not to open an email containing malicious code that will allow hackers to intercept the next download from Dominion. Once they’ve got the rural county’s ICE software, they can use that knowledge to interfere with the next election in any other county that uses Dominion software.

This one problem could easily be fixed, as Freiberg demonstrated. Dominion ICE users could simply refuse to download software over the internet, and work with their vendor to find a different way.

Less easily fixed is Dominion’s way of doing business. Why did Freiberg even have to ask for an alternate method of obtaining the software? Does Dominion itself understand the risks to election security and voter confidence?

And it’s not just this one slip-up. Independent observers and experts have expressed serious concerns about the design of the system. Other serious concerns are:

  • The ICE system is designed in a way that would allow someone to program it to print additional votes on a ballot after the voter has cast it. This feature renders elections conducted on these machines unauditable, because the ballots were not secured from alteration after leaving the voters’ hands.
  • The ballot-printing feature of the system records voters’ selections in the form of barcodes printed on the ballots, which the tabulator reads when it counts the votes. This means that the voters are unable to verify that the counted votes are the ones they intended to cast.
  • The ICE system incorporates a feature known to security advocates as “permission to cheat.” A voter who uses the touchscreen to mark his or her votes can choose to have the machine count the votes and drop the printed ballot into the bin without the voter’s review–essentially giving the computer programmers permission to cheat. Security advocates (and common sense) insist that voters MUST verify the integrity of the printed ballot if election results are to be trustworthy.

Arguably more than any other voting system, the Dominion ICE is the target of voter concern, even outrage. The most direct, immediate solution is for candidates and voters to demand that no more counties buy the ICE system, and to demand that their election officials who already use it follow the Fond du Lac county clerk’s example and refuse to download software over the internet.

Beyond that, we need to take action to make Dominion take security seriously or to prohibit use of their products. The ICE system could be decertified at either the state or federal level, and federal legislation could prohibit the use of voting systems capable of changing voters’ ballots after they have been cast. 

* Door, Fond du Lac, Grant, Green, Ozaukee, Racine, Trempealeau (one municipality), Vilas, Walworth, Washington, Waupaca (four municipalities), and Winnebago Counties

Projected Ballot Counting

Paper ballots can be manually counted in different ways–sort by candidate and then count the ballots; stack the ballots into groups of 20 and 100 and then have counters mark tally sheets as they go through the stack one-by-one; and more.

Affordable technology–a simple digital camera hooked up to a projector–can beat all these methods on each of the four attributes of a good manual-counting method.

1.  Ballot security.

Ballots must not be altered by the manual count.  Sorting and stacking methods require the ballots to be handled several times, by several people, and moved around tables. When ballots are projected, only one person needs to handle the ballots, only once, and can keep them on one table, in full view.

2.  Accuracy.

In a manual count, accuracy is established with redundant counts—two or more people must agree on each vote, reconciling any disagreement.  When counters make errors in sort-and-stack or tally-sheet methods, finding and reviewing the problem ballot can take a lot of time and ballot-handling. With projected ballots, everyone sees the same vote at the same time, so ambiguous votes can be reconciled when they are first encountered.

3.  Speed.

Faster methods of manual counting help to restrain costs, because labor is the biggest cost. Quicker counting also makes the task more pleasant for both counters and observers. Projected-ballot manual counts have accurately counted votes in one race at a rate of 100 ballots every four minutes, including time to stop to compare paired counters’ totals and resolve any differences. Depending on ballot design, two races could go just as fast.

4.  Transparency.

The value of a manual count depends upon how much trust it produces in candidates and voters. In traditional manual-count methods, observers cannot see ballots well enough to verify for themselves that the votes are being counted accurately and honestly. 
When the ballots are projected, observers see exactly what the official counters see. In addition, because projected-ballot counts require no ballot-handling by the counters, observers can be drafted on the spot as official counters–powerfully counteracting any distrust.

A tally sheet completed in full view of all counters and observers serves as a record of the manual count.

A pdf document containing step-by-step instructions is here.

I wish you had seen this.

The Wisconsin Elections Commission met today, and I stayed for most of the agenda.

One agenda item had to do with fixing the snafus that caused a voter-registration list maintenance effort in 2017 to incorrectly ‘deactivate’ thousands of validly registered voters. (You may have heard such efforts described as ‘purges,’ a relatively pejorative term that is fitting whenever voter-list maintenance is used as a voter-suppression tactic.)

Among other things, so many voters were incorrectly removed from the registration lists that poll workers for the past several elections have had to work with two sets of poll books–the regular one for unaffected voters, and a supplemental list of voters who had been struck from the rolls but who would be allowed to vote if they showed up on Election Day and attested that they had not, in fact, moved.

There are dozens of reasons, it turns out, why State of Wisconsin computers got confused about whether these voters had moved. They have to do with things like registering a vehicle with your personal name but your business address, or buying a car for your college student in La Crosse and registering it there instead of where you vote. I won’t go into all the details. If you’re curious, you can read the staff report starting on page 72 of this document.

I spend a lot of time reading about election-integrity problems in other states. That means I read about a lot of skuzzy partisan machinations.

I also spend some time talking with local election officials. That, unfortunately, exposes me to much whining, excuse-making, buck-passing and “no law says I have to” attitude.

Here’s why the WEC discussion impressed me so much that I had to come home and write this blog post.

The discussion was pure, unadulterated problem-solving, start to finish. No one was looking for a partisan angle or opportunity. Not one single commissioner or staff member was whining. No energy was wasted on self-protective defensiveness, or on denying or minimizing the problems. I heard no attempts at buck-passing, no excuses.

Unlike what I hear when I talk to many local election officials about vote tabulation, no one at WEC was pointing out that statutes require them to do the work but don’t require them to do it right. It didn’t seem to cross any Commissioner’s mind to avoid their managerial obligation to detect, analyze, and correct problems until someone passes a law forcing them to do that, and paying them extra for it.

WEC commissioners and staff were straight-up committed to discovering the extent of the problems and what caused them, and to making sure they never happen again. Commissioners asked staff for hard data on error rates, and made sure that staff are not sending any more deactivation notices until the problems are fixed. Staff, for their part, were as committed to getting past problems corrected and future problems averted as the Commissioners were.

This is what responsible election administrators look like.

I wish all voters could have seen what I saw today. And I wish some reporter would write about it when good work gets done.

An Illustrated Introduction to Risk-Limiting Audits

Posted by Karen McKim · December 19, 2018 2:21 PM

December 19, 2018 — “As the secret ballot transformed elections in the last century,” said Joseph Hall, Chief Technologist for the Center for Democracy and Technology, “risk-limiting audits are going to transform elections in this century.”

 In a few years, Americans will look back, aghast, at our current election management. We will be amazed that we ever trusted vote-tabulating computers so much that we routinely declared winners without checking results for evidence of fraud, glitches, or human error.  We will consider routine verification to be an indispensable part of managing elections, just as cash-register reconciliation is now for managing the corner convenience store.

In preparation for that day, it’s time to understand: What is a “risk-limiting audit”?

First, a risk-limiting audit is not a specific set of steps or statistical calculations. Like “home-heating system,” the term describes a function, not a method. If a system heats your home, it’s a home-heating system. If it doesn’t, it’s not. A wood-burning stove is a home-heating system. Electrified tile floors are a home-heating system. Triple-pane windows and attic insulation are not.

A risk-limiting audit is any review that imposes a precise limit, such as 10%, on the risk of certifying incorrect results in the event that Election-Night results identified the wrong winner. Any review that accomplishes that is a risk-limiting audit. If it doesn’t, it’s not. For example, pre-election voting-machine tests and hand-counting to verify a few voting machines’ results are good. But even when completed correctly, they do not precisely limit the risk that officials will detect and correct any outcome-altering miscounts.

(Though it’s not part of the official definition of risk-limiting audit, I’ll mention the other side of the coin. In the event that Election-Night results identified the right winner, a risk-limiting audit does not reduce the risk that officials will certify the wrong one. That risk stays at zero.)

You might be surprised that statistical analysis is not a required feature of risk-limiting audits. A full recount uses no statistical methods and if done correctly, limits the risk of declaring the wrong winner to zero. Therefore, it’s a risk-limiting audit. But full recounts require too much effort to be used routinely. So to reduce the time and effort needed to confirm elections, most types of risk-limiting audits use random sampling for selection and statistical processes for analysis.

Another feature of risk-limiting audits is manual inspection of voter-verified paper ballots. Until some as-yet-uninvented technology comes along, we can verify the computers’ verdicts only by comparing them against something that didn’t come from a computer. That is, we need direct human observation of the paper ballots that were marked, or at least verified, by the voters themselves.

Finally, the word ‘audit’ doesn’t mean what you probably think it means. Outside elections, independent auditors perform audits after auditees have completed the work. In contrast, a risk-limiting audit is completed by the responsible officials as part of their work of certifying election results. A post-certification review might provide useful information for the next election, but it cannot limit the risk that the wrong winner was certified in this one. Risk-limiting audits probably should have been called ‘risk-limiting reconciliation’ or ‘risk-limiting verification,’ but it’s too late to change that now.

Try it yourself…

In December 2018, national election-security leaders came together at an Election Audit Summit in Boston, organized by the MIT/Caltech Voting Technology Project. Dr. Philip Stark of the University of California-Berkeley, who originated the concept of risk-limiting audits, led participants through a demonstration. The instructions below are adapted from Dr. Stark’s handout, Audit Simulation for Auditing Summit

In brief, the exercise uses playing cards to represent ballots containing votes for Black or Red. The cards are sorted into piles representing precincts; a sample is randomly drawn from across all participating precincts. Each card either builds or reduces confidence in the Election-Night results, until a pre-determined acceptable level of confidence is achieved—or is not. To do this exercise, you’ll need:

  • two decks of playing cards;
  • scratch paper;
  • a pencil; and
  • a random-number generator.  

Note: the statistics in this exercise are realistic but not precise; they are for illustration purposes only. A real election audit would use a sample size and confidence level calculated for the results being audited.

Example #1: When Election-Night results are correct

The first illustration will show how a risk-limiting audit plays out when Election-Night results identified the correct winner.

1. Get two decks of playing cards. They don’t need to be the same design, but the same shape and size will make them easier to work with. From one deck, remove the jokers and set the hearts aside. This leaves 39 cards in this deck—26 black ‘votes’ and 13 red.

2.  From the other deck, remove the jokers and set aside both the hearts and diamonds. This leaves 26 cards in this deck, all black.

3. Shuffle all 65 cards together, but not thoroughly. Actual ballots will not be thoroughly shuffled; your cards don’t need to be, either.

4.  Separate the cards into six stacks to represent precincts. You don’t need to make them the same size. In the real world, some precincts have more voters than others. Label the stacks “Precinct A,” “Precinct B,” and so on up to “Precinct F.”

5.  Count the cards in each precinct consecutively and write the numbers on the labels. For example, if Precinct A has 12 cards, you will write “1-12” on that label. Then start Precinct B’s count with 13, so that Precinct B’s label will be something like “13-23.” Precinct C will be something like “24-35,” and so forth. If you’ve counted correctly, the last card in Precinct F will be 65. 
Some vocabulary: A list of precincts indicating the number of ballots in each (e.g., Precinct D has 13 ballots) is a “ballot manifest.” When you assign a unique number to each ballot (e.g., Precinct D contains the 36th through the 48th ballot), it becomes a “ballot look-up table.”

Now, imagine you’re the official who is responsible for certifying this election. You know the possibility of an outcome-altering Election-Night miscount is remote. Nevertheless, you want to: 1) deter fraud in future elections, 2) demonstrate to the voters that local elections are secure against even remote risks; and 3) make sure you don’t miss even a remote possibility of declaring the wrong winner.

So you’ve decided to give yourself at least a 90% chance of detecting any outcome-altering miscount before you declare the official winner. In other words, you have decided to impose a 10% limit on the risk that you will fail to notice and correct any such miscount. (You could choose a different level of risk, if you wish.) If the machines identified the correct winner, there is a 0% chance an audit will reverse that.

Initial Election-Night results indicated that Black got 80% of the vote and Red got 20%. While you haven’t looked at any ballots yet, you know how many were cast.  Using that information, you consult with a statistician or the risk-limiting audit website to find out how big your manually counted sample needs to be to confirm the right winner or to detect the wrong one.

In a real risk-limiting audit, you would be told a specific number of ballots to draw for the first sample—up to a few hundred, depending on the margin of victory and the number of ballots cast in the race. Your statistician or the RLA website could also generate random numbers for you, to determine which ballots to draw for the sample. For this demonstration, let’s imagine you were told to select 10 ballots.

6.  Create a score sheet with columns for the random number, the color of the card, a confidence score for each card, and a running confidence total.

7.  Generate ten random numbers between 1 and 65. Write them in the first column. These are the ballots you need to inspect. 

8. Then find each card. For example, if the first randomly selected ballot was #38, you would check the ballot look-up table and see that card #38 is the third card in precinct D. Look at that card, record its color in the second column, and replace it.

  • If the sampled card was black, it builds confidence that the preliminary results were correct when they identified Black as the winner. Note +5 confidence points for that card in the third column, and add 5 points to the confidence total in the fourth column.
  • If the sampled card was red, it reduces confidence in the preliminary results. Note 10 confidence points in the third column, and subtract 10 points from the confidence total in the fourth column.

9.  When you’ve recorded the color of each card in the sample, look at your total confidence score. If it is 25 or higher, you have statistically confirmed, with 90% confidence or more, that no Election-Day miscount identified the wrong winner. You can end the audit and certify the results.

The photo below shows that this audit could have stopped after the eighth card, because the confidence score reached 25 at that point. On average, an audit like this would need to inspect 14 cards to reach a confidence level of 25—if, that is, the Election Night result was correct.

If your confidence score is less than 25, select another random ballot, and another, until the total confidence level reaches 25. (Or until you realize that you messed up the first two steps of this exercise and are working with a deck in which there are actually more red than black cards.)

#2: When Election-Night results are incorrect

The second part of this exercise shows what happens when Election-Night results were wrong.

1.  Retrieve the red cards you set aside, so that you are using all 104 cards from both decks. Shuffle them together. Again, this does not need to be a thorough shuffle. Some ‘precincts’ can be mostly red and some mostly black.

2.  Separate the cards into 9 stacks of differing sizes to represent precincts. As in step 5 above, count the cards in each stack to create a ballot look-up table.

3.  In this exercise, we know the election was a tie. But in a real election, we would not know that, because the computers told us that Black won, and we have not yet inspected any ballots. So we would give the same information to the statistician or RLA website that we did in the previous example—that is, an 80% victory for Black. Given that situation, the instructions we receive would be the same: Select a random sample of 10 ballots by generating 10 random numbers—this time, between 1 and 104, because we have more ‘ballots.’

4.  As in the first exercise, use the ballot look-up table to find each card selected for the sample. Note the color of each and confidence points on the tally sheet. Check to see whether the total confidence score reached 25. In this case, it did not.
It is possible that your first sample reached a confidence score of 25 or more. If so, your audit incorrectly confirmed a miscounted election. This can happen—statistical confidence is not the same as rock-solid certainty. A 90% confidence target means that 10% of the time it will be wrong. To calm your nerves, think of this from the point of view of election thieves who see a 90% chance that their handiwork will be noticed and corrected.

5.  If the total confidence score is less than 25, you have not yet confirmed the Election-Night results, so you will need to expand the sample by inspecting more random ballots.


When the Election Night results are wrong, the more ballots you sample, the lower your confidence level will sink. As shown in the photo, as more and more ballots are inspected, it becomes more and more apparent that the preliminary results are just not right.

Election officials who conduct risk-limiting audits of real elections adopt written policies that address this possibility. A sensible policy, for example, might dictate that the audit will stop if it fails to confirm the outcome with two additional samples, and the effort will instead be put into a manual count of 100% of the ballots.

About sample size, statistical confidence, and emotional confidence

One concept should now be clear: A random sample of ballots is a miniature version of the entire election.  The same winner will emerge from both–if both are accurately counted.

In the first exercise above, Black had more votes in the whole set of ballots from which the sample was drawn. As a result, more of the randomly selected ballots contained votes for Black than for Red. In the second exercise, Black did NOT have more votes than Red, and so we could not confirm a Black victory no matter how many ballots we randomly drew.

In other words, when preliminary election results have identified the correct winner, inspection of a relatively small number of randomly selected ballots provides strong evidence of accuracy. Conversely, if preliminary election results identified the wrong winner, inspecting a random sample of ballots will reveal the problem before officials certify the election.

Once we see that random samples reflect the true results, the next question is what size sample works best. Smaller samples reduce work, but increase uncertainty. Larger samples provide more confidence, but are more work.

This demonstration started with samples of 10 cards, which in the 65-card ‘election’ was a little more than 15% of the ballots. In a real election audit, the initial sample size would not need to be anything close to 15% of total ballots, particularly if the preliminary results indicated an outcome as decisive as this one. 

To work through a real-life example, let’s look at the 2018 race for US Congress in Wisconsin’s First Congressional District. This reasonably close and hotly contested race filled the seat being vacated by former Speaker Paul Ryan. The actual results were: 325,003 total ballots; 177,490 votes for Steil; 137,507 for Bryce; and 10,006 for Yorgan.

When you plug these results along with a 10% risk limit into the online tools for ballot-polling RLAs, statistical operations built into that tool predict that you will likely be able to confirm Steil’s victory, if Steil actually won, with an initial sample of only 301 ballots. That’s only one-tenth of one percent of the 325,003 total ballots. If Steil did not truly win, auditing 301 ballots would produce results more like the second example above–forcing officials to keep expanding the sample until it was obvious that the Election-Night results were incorrect. (Instead of using +5, -10, and 25 total points as indicators of confidence, officials would have counted the votes in the sample and could have used the online tools to assess the results. See the section titled “Should more ballots be audited?” at the bottom of this page.)

If election officials did not believe that 301 ballots would provide voters with enough subjective confidence in the result (as opposed to statistical confidence), they could have selected a smaller risk limit. In this congressional election, a 5% risk limit would have needed an initial random sample of 389 ballots; a 1% risk limit, a random sample of 594 ballots. Or, election officials could adopt an audit policy that every initial sample will contain either enough ballots to support a 5% risk limit, or 1,000 ballots, whichever is greater.

Other lessons from this exercise

This exercise highlighted risk-limiting audits’ tightly focused purpose—to detect and correct any outcome-altering miscounts. This purpose is critical for election security and for voter confidence, but does not solve all problems. Election officials must perform other types of reviews to determine the cause of any miscounts and to monitor other important issues, including:

  • The presence of any miscounts that may have disenfranchised some voters without affecting the outcome;
  • The accuracy of any single precinct’s or voting machine’s tabulation;
  • The quality of any of the processes that determine which ballots were cast and accepted, such as issues with voter registration or ID, or whether all and only valid absentee ballots were accepted and counted.

In addition, this exercise simulated only one type of risk-limiting audit, known as a “ballot-polling audit.” Depending upon the size of the election, the type of records created by the voting-equipment, and other factors, election officials might decide to use a different type of RLA, such as a “comparison risk-limiting audit” or a Bayesian audit. Election officials do not need to read and digest the scholarly articles. Federal officials, staff in jurisdictions that have experience with auditing, and other experts are willing to help local election clerks understand the options well enough to make the right choices and get started with election auditing. A local election official can likely find useful help as close as the nearest local government official who has expertise related to auditing or quality assurance. 

This exercise also probably raised some implementation questions: How can election officials draw a random sample in a election for which ballots are stored in sealed bags across hundreds of municipalities? How do you know how big your sample needs to be if you want to verify the outcomes in two or more races? Election officials who have started with risk-limiting audits have tackled these questions, and more solutions are being worked out with each new election. The solutions are not always easy or obvious, but local election officials who want to try their hand at risk-limiting audits need only to ask those with experience.

Finally, this exercise demonstrated that even risk-limiting audits might, on occasion, fail to detect miscounted election results. A 90% chance that serious fraud will be detected and corrected is the same thing as a 10% chance it will not be.

That highlights the need to keep two other facts in mind. First, a 90% chance of detecting fraud is better than the 0% chance that non-auditing election officials and their voters now tolerate. Second, the audits’ greatest value is, arguably, deterrence. When would-be election thieves contemplate a 90% risk of getting caught, there’s a good chance that election officials will have no electronic election fraud to detect.electionscounting votesWisconsinrisk-limiting auditsdemonstratio