Photo: Michigan election officials assess the results of a manual count of a sample of ballots for a risk-limiting audit in 2018. Photo credit: Berkeley Institute for Data Science, UC-Berkeley
Think of “risk-limiting audits” as low-effort exit polls.
Exit polls determine who won by asking randomly selected voters “Who did you vote for?” Risk-limiting audits work on the same principle to confirm the correct winners, but they do not involve talking to voters. Instead, RLAs pose the question directly to randomly selected paper ballots.
Either way, a small sample can provide statistical proof of who really won the election, independently of the vote-counting computers.
No one in Wisconsin now does risk-limiting audits. Sometimes local officials spot-check a few randomly selected voting machines, but to any sensible risk manager’s horror, none have any specific plans for what they will do if any of those random voting-machine audits ever detects a serious miscount. Risk-limiting audits, in contrast, always resolve any problems they detect.
The Wisconsin Elections Commission has shown some mild interest in the possibility of using outcome-verifying audits to secure our elections. Every sensible Wisconsin voter should be pressuring them to do more. Here is the memo I submitted to WEC in December 2019, to help them understand how very practical this effective safeguard could be.
There’s no one correct way to do a risk-limiting audit. Our election officials could sample individual ballots (less work) or entire polling places (more work). They could do nothing more than confirm the correct winner in one race (less work) or they could answer other questions at the same time (more work).
A risk-limiting audit of a statewide election in Wisconsin could be this easy and cheap:
1) After they close the polls on Election Night, poll workers would record how many ballots they seal into each bag. Using this information, the municipal clerk would create a “ballot manifest” (e.g., City of Abbotsford: Bag #1 – 234 ballots; Bag #2 – 122 ballots).
It’s unlikely anyone has ever counted, but a fair guess is that a big election produces around 4,750-5,000 sealed ballot bags statewide. One bag can contain a maximum of around 300 ballots but might contain fewer than 10.
2) The day after the election, every municipal clerk would send their ballot manifest to the Wisconsin Elections Commission. The WEC could create an online reporting form to make this task easy and quick. It wouldn’t need bullet-proof security if the municipal clerk also mailed a hard copy of the manifest to WEC, and WEC staff later verified them against each other.
3) The WEC would then assign a number to every ballot in the state. For example, ballot numbers 1-234 would be assigned to the first bag from the City of Abbotsford; numbers 235-356 to the second bag, all the way up to the last bag from the Town of Yuba, which might be assigned the numbers 2,673,149 – 2,673,308.
4) WEC staff would examine the preliminary election results for the statewide races and enter the results for the closest race into a statistical tool that has been endorsed by the American Statistical Association, tested, and used in other states. This would generate a sample size for the audit.
The size of the sample depends upon the Election-Night margin of victory. If the margin is large or normal, the sample size will be small. For example, the 2018 contest for the US Senate was neither close nor a landslide: 55.4% to 44.5%. A risk-limiting audit of that race would have needed an initial sample size of only 401 ballots across the entire state. However, officials could choose to select a larger sample to provide voters with ’emotional’ confidence in addition to statistical confidence.
An extremely close election such as the 2018 Governor’s race (49.5% to 48.4%) would have needed an initial sample of 37,841 ballots (out of almost 2.7 million cast). But it’s these races that officials legitimately need to be most careful about, and it’s the very close races that, when left unaudited, provoke the most candidate resentment and voter suspicion.
Wisconsin election officials have already demonstrated they can handle larger sample sizes. For comparison, the voting-machine spot-checks conducted after the November 2018 election required officials to count votes from 135,712 ballots — more than 3.5 times the number of ballots they would have needed for a risk-limiting audit. But because of the way WEC selected that sample and their instructions that auditors ignore voter intent, that effort did not confirm the correct winner in any race.
Wisconsin election officials counted 135,712 ballots in the random voting-machine spot-checks after the November 2018 election, but used a method that did not confirm the winner in any race.
A risk-limiting audit of the same election would likely have verified the correct winners in the statewide races with only 37,841 ballots.
And because races from the same ballot (as those two races were) do not need separate samples, a risk-limiting audit could have verified all the statewide contests on the ballot in that election–an accomplishment of enormous value to election security and voter confidence.
5) WEC would randomly select ballot numbers and then use the statewide ballot manifest to identify the bag in which each of the selected ballots is stored. For example, if ballot #284 turned up in the random sample, the WEC would know it is in the second bag from the City of Abbotsford. If the random selection turned up ballot #2,673,193, they would know it is in the last bag from the Town of Yuba.
6) At this point, WEC could ignore the hypothetical numbers they assigned to each ballot and tell the municipal clerks only the number of ballots to be randomly selected from each bag.
For example, the WEC would tell the City of Abbotsford clerk to randomly select one ballot from the second bag. The instructions for random selection could be something like: “In the presence of observers, pull the ballots out of the bag, set them in a stack on the table, let an observer from each political party cut the stack several times like a deck of cards, cut the stack two more times, and select the ballot at the bottom of the last cut.”
Other methods could be prescribed for jurisdictions that use machines that print flimsier forms of paper ballots.
7) The municipal clerk would display the selected ballot to the observers; fax it to the WEC; mark it with red ink indicating it was the ballot selected for the audit; put it on the top of the stack of ballots; and reseal the bag.
8) The WEC would conduct a publicly observed manual count of the faxed ballots and enter the results of that count into the standard risk-limiting audit formulas. If the proportion of votes for the Election-Night winner in the manual count is close enough to the proportion reported on Election Night, the result is confirmed. The audit would be concluded and the county canvasses could conduct their certification process as normal.
If the proportion of votes for the Election-Night winner differed too much, an additional sample would be drawn and counted. That process would be repeated until statistical confidence in a winner was established.
The WEC would need to adopt policies to govern what will happen in the rare event that the sample has to be expanded more than twice, or if the confidence level declines as the sample is enlarged. Likely, WEC would stop the audit, declare a lack of confidence in the preliminary Election-Night results, and order a full recount on its own initiative.