What do we want: “Not one wrong vote” or “Maybe a few thousand is okay”?

In brief:  Our election officials tell us that every single valid vote and not one single invalid vote should be counted. But if you watch closely, you’ll notice that election officials tend to tolerate any size or type of miscount unless it seems to affect the outcome.
Well, which is it? Do we want 100% accuracy, or do we want only approximate election results that probably identify the right winner? Until we face the need for clear and consistent standards, we cannot hold our election officials to any.

(Updated with additional information from a June 2017 report from WEC staff–see end of blog post.) 


May 30, 2017 — During the 15 years I supervised investigations for Wisconsin’s legislative oversight agency, I learned the value of several specific thinking skills. One was to be clear about expectations.  

Auditors call them ‘standards’. Investigations start when someone reports that something is not as it should be. A good investigator’s first step is to figure out: What is it that should be?  What would things look like if everything was working right?

If, for example, the complaint is “This permitting process takes too long”, we need to know how long people expect it to take.

This habit of noticing standards—or their absence–spilled over into the rest of my life. It’s useful. You probably know that you’re less likely to be swayed by a salesperson if you’ve already decided how much you want to spend before you set foot in the store.

And standards keep us heading in the right direction. As the Cheshire Cat told Alice, which way you need to go from here depends a good deal on where you want to get to.

The way things are now, however, paying close attention to the standards we set for our elections can make a person dizzy.

Sometimes we apply a strict standard:  Not a single wrong vote. Not one. 

According to Chris Ott, Executive Director of the Wisconsin ACLU, “We know precisely the number of eligible voters that is too many to keep away from the polls: even one.”

He was talking about Voter ID, and how it keeps some eligible voters from the polls.

Those who fuss about ineligible voters proclaim the same single-vote standard. In April 2015, I observed a Wisconsin county board of canvass as they decided what to do about a ballot cast by a voter who lacked ID. She had been given a provisional ballot. Those ballots are supposed to be sealed in an envelope and counted only after the voter returns with identification. However, the voter naively fed it into the voting machine right away, and it was counted.

There were 234,680 other ballots in that election. But the officials decided that if the voter did not provide identification by the deadline, they were going to go after that one rogue ballot. They were going to unseal the ballot bags, find it, and discard it. They were not willing to tolerate even one wrong vote.

And on those rare occasions when officials find evidence of voter fraud, they take action.  In 2014, a Wisconsin audit reviewed voting records in statewide elections from February 2010 through April 2014 and found that, across four years and 16 elections, a total of 33 felons might have cast illegitimate votes. The legislature directed the state elections agency to refer the cases to district attorneys for possible prosecution.

This single-vote standard is also applied to voting machines. Federal standards require no more than one miscounted vote in every 500,000 from perfectly marked ballots, when counted in laboratory tests.

In real life situations, modern paper-ballot systems have achieved 99.99% accuracy even when the ballots contain some poorly marked votes.  But in 2012, one Texas county thought “hand-marked paper ballots …open the door to ambiguous voter intent.” The possibility of one miscounted vote in 10,000 was too much for them, so they threw away their paper ballots and bought touchscreen voting machines instead.

Other times we apply a lax standard: A few thousand wrong or miscounted votes isn’t anything to worry about.

In the 2016 recount of Wisconsin’s presidential election, one county’s board of canvass discovered more than five dozen valid absentee ballots had been left uncounted on Election Day.  And the errors didn’t stop there. That county’s officials determined they and their voting machines had originally miscounted 1,475 votes—either not counted them when they should have, counted when they should not have, or counted them for the wrong candidate. Then, when reporting the recount results, they accidentally recorded 21 votes in the wrong candidate’s column. They did not correct that error when informed of it. (1)

Commenting after the recount, the county clerk said he was “extremely proud.” He made it clear he thought the recount had been unnecessary. Checking accuracy when the race is not close, he wrote, is “really quite an insult to suggest that we don’t know how to do our jobs.”

News media coverage of the recount reflected the same relaxed standard.  In Marinette County, the recount revealed that one city’s voting machines had failed to detect votes on 291, or 24%, of the ballots. This miscount was obvious to both municipal and county officials on Election Night, but they considered it tolerable and certified results without correcting the error. Only the recount forced correction.  No reporters considered the incident worthy of investigation or reporting, and it received no coverage.

The Wisconsin State Journal’s headline made its standard clear: “Recount found thousands of errors, but no major flaws.”

So which accuracy standard should we use—exact or approximate?

Readers familiar with my work may be surprised that I don’t think we should pursue 100% accuracy.  Until we can find someone other than humans to run our elections, perfection is not within our grasp. Reaching for it throws us off balance. For example, when the Texas county threw away voter-marked paper ballots because of a tiny miscount rate, it threw away its ability to detect miscounts of any size because it no longer had an auditable record.

The whole voter ID mess is the worst illustration of what happens when we seek perfection. Even without ID requirements, our registration system prevents all but insignificant voter fraud. And yet states with voter ID are spending millions to go after the remaining, minuscule problem. They are inconveniencing every honest citizen who casts a vote, and preventing thousands from voting at all. All in the vain pursuit of perfection.

So I’m not on board with an “every single vote” standard of accuracy.  But when our election officials and media yawn at thousands of miscounted votes, I’m not willing to say “Mistakes are okay,” either.

A complete lack of standards leaves the door open for wild inconsistency. I told you about one county where officials were willing to track down and discard even one illegitimate vote out of more than 234,000. And I told you about a county where standards were so lax that when they discovered 1,475 miscounting errors, the county clerk said he was insulted that anyone would request verification.  Both stories are from the same county, the same clerk, the same board of canvass—Dane County, Wisconsin.

That’s what happens without standards. Officials can pick and choose their own targets, grabbing one before Election Day and a different one after. They can apply one standard to voters’ errors and a different one to their own. Without standards, voters, candidates, and reporters—and conscientious officials—have no solid ground to stand on when they try to assess the quality of the elections.

If we had a clear standard—say, 99% confidence that the election results identified the correct winner—we could make better judgments about where to spend our effort and election resources. We could set a threshold for voter fraud, and establish that it is not worth millions of dollars to prevent a few people from voting twice. We could tell our election officials that no, it’s not okay to declare election results final until after you’ve corrected the voting-machine output that missed one in every four votes.

And by the way, should we count write-in votes?

It’s not just a question of picking a maximum acceptable error rate.  Wisconsin hasn’t yet settled the question of how—or even whether—to count write-in votes.

For those who believe elections need to do no more than identify winners, write-in votes are not worth counting. Some election officials appear to be in this group. In last November’s presidential election, Wisconsin officials counted only 84.3 percent of the votes for registered write-in candidate Evan McMullin until the recount forced a do-over.  That is, more than one in seven voters’ expressed preference for McMullin was simply ignored when officials first tabulated and certified the election results.

Bernie Sanders’ voters fared even worse. Wisconsin law currently does not require officials to count votes for unregistered write-in candidates.  The state elections agency allows counties, at their option, to report the total number of such votes in a miscellaneous category called ‘scattering.’ But not all counties do. So 24 of Wisconsin’s 72 counties did not report the number of miscellaneous write-in votes in the original results, the recount, or both. In these counties, write-ins for unregistered write-in candidates were literally—and legally—treated as if the ballots were blank. No one has any idea how many voters were so frustrated with the nominees that they were willing to go to the polls to express their feelings.

For those who believe election results need to reflect the voice of the people, that’s not okay. Some voters may be content to have the voice of the people constrained to saying only one of two things: either “I like the Democratic candidate” or “I like the Republican candidate.” But others want an opportunity to say that they want someone else, and to have that voice heard.

If and when we want to improve the accuracy of our elections—or even talk about it productively—we’re going to have to face and resolve these questions.

(1) The author detected the error—21 votes added to Cherunda Fox’s total for the Village of Waunakee—while reviewing the recount findings. An email exchange with the Village Clerk in March 2017 confirmed that the votes should have been added to Evan McMullin’s total. A check of official results on the state’s website on May 28, 2017 confirmed that the error had not been corrected—and likely will not be, because the recount results have been certified final and are no longer subject to correction.

UPDATE, June 20, 2017

At today’s meeting of the Wisconsin Election Commission, staff presented a report on the 47 voting-machine audits–something different than recounts–that were performed in accordance with s.7.08(6), Wis. Stats. following the November election. My detailed description of how these audits work is here.

Instructions for the audit (page 6 of the memo) seem to indicate the audit standard is “not one vote must be miscounted.” However, the memo is not specific about the number of votes in any of the audits, which gives us a sign that the officials performing the audits and reporting the results were applying a less precise standard. 

One of the villages randomly selected for audit, the Village of Hortonville in Outagamie County, could not reconcile the results of its audit hand-count with the Election-Night results they reported and certified. (Outagamie County uses the Optech Eagle, and did its recount by machine, so the electronic miscounts must have merely been replicated in the recount, not caught.)

However, the report does make it clear that after separate attempts by the municipality, county, and state to reconcile the different vote totals, “This exercise did not produce a result that allowed staff to understand how the Optech Eagle treated these ballots with confidence.”

Now, back to the point of this blog post: What is the implicit standard that our election officials applied in this case? ‘Not one vote’, or ‘miscounts are okay’?  A key statement is on page 11 of the memo:

“The analysis of the performance of the Optech Eagle identified a significant limitation of the equipment, but the Optech Eagle performed as expected during the 2016 Presidential Election.”

Read that again to make sure that phrase sinks in: “A significant limitation…as expected.”  That’s the standard for accuracy that is applied by our election officials to Wisconsin’s voting-machines.

Report on the Dane County Canvass: 99.6% unverified

April 14, 2016 at 1:29pm — I observed Dane County’s post-election canvass of the April 5 election results from start to finish this year–all 19 hours over 4 days.

No surprises: As usual, both the municipal and county canvasses checked and double-checked to make sure the right number of BALLOTS had been counted. However, the Board of Canvass (County Clerk Scott McDonell, Democratic Party representative Gretchen Lowe, and Republican Party representative Joyce Waldrop) certified Dane County’s election results at around 3:30 PM on Wednesday, before any one had done anything to verify that the correct number of VOTES had been counted.

The votes of 234,681 Dane County voters (99.6% of the total) were certified based only on unaudited computer output. Vote totals are now no longer subject to change or correction.

The other 859 ballots were late-arriving absentee ballots and approved provisional ballots, which had been publicly hand-counted by the municipalities. The last two days of the county canvass were devoted to making sure the votes from those 849 ballots were added to the correct candidates’ totals.

Got that? Half the canvass effort to ensure the accurate counting of only 0.4% of the votes.

Over four days, County Clerk McDonell maintained the minimum transparency required by law. Any observers who were already familiar with the statutes and GAB guidance for county canvasses (that would be me) could follow along reasonably well, but anyone else would have been out of luck in terms of understanding what the canvassers were doing or why. McDonell provided no written procedures or standards–not even to the members of the board. Neither did he explain what they were doing as they went along; allow questions from observers; or provide observers with copies of anything the canvassers were looking at, or make it visible to them in any way such as by projecting it on a screen.

“Just guess” was the unspoken message to the public. Finally, he restricted any public comment to five minutes at the end of the four-day-long meeting. 

It got this bizarre: At the end of the four days, I asked if I could ask a question and was told I could make a five-minute statement and that was it.  (McDonell claimed that to answer a question for the public would be a violation of open meetings law.)

So the official public comment at the county canvass started with this awkwardness coming out of my mouth: “I noticed an agenda-less canvass meeting on the county calendar for 10 AM on April 20. I assume that is the digital-image audit you’ve been promising. I hope you will let me know if I am right or wrong in that assumption.” 

I’m not making this up: McDonell didn’t even nod yes or no.  And when Waldrop wanted to respond to my comment, he wouldn’t allow that, either. 

This created something of a Mad-Hatter-Tea-Party feeling to the event, since it was basically just four of us sitting around a table in a conference room in the City-County building, sharing Girl Scout cookies from McDonell’s daughters. The canvassers and I would chat whenever McDonell left the room, but when he was present they had to pretend I wasn’t there, as McDonell himself did.

I left them with a letter, which I’ve uploaded here.  It’s kind of wonky–I wanted to address them as professionals who know and care what words like ‘risk,’ ‘prioritize’ and ‘verification’ mean. I could see that at least Lowe was reading it carefully, and she asked me a few sensible questions after the meeting adjourned. The main points of the letter are:

  • they spend most of their time addressing risks that are much more remote than the risk of electronic miscounts, or that address no risk at all–such as reviewing vote totals in uncontested races for which it would be impossible for them to certify the wrong winner; and
  • they also spend time on tasks that don’t need to be completed before they certify the election results, such as discussing individual municipalities’ Election-Day practices for keeping track of the number of voters.

And yet they tell us they have no time to check the accuracy of the computer-generated vote totals–which cover 99.6% of the votes.

My request to them wasn’t anything dramatic: I simply urged them to consider risk and timeliness when they decide what to do during the canvass, and told them if they thought about it that way, it would be obvious that verification of the computer output is more important than most of what they are doing now.

As I sat listening to them recite numbers for four days, I visualized the following graphic, which shows:

  1.  The steps by which our votes are turning into final election results;
  2.  The parts of this process that are verified by the current county canvass procedures; and
  3.   The parts of the process that are verified by the type of audit we’ve been demonstrating in our citizens’ audits.

Local clerks’ excuses for not talking about election audits

December 20, 2015  — Before I retired, I worked in quality assurance for a program that delivered health care in clients’ homes. It was a human-services government program, so the provider agencies never had more than a a shoestring budget. Rarely was there only one right way to do anything.

Whether because of or in spite of that, most managers were willing to think and talk about quality improvement.

But there were always a few mangers who reflexively insisted, “Everything is already as good as it can possibly be, and I don’t need any new ideas.” 

Now that I’m working in election integrity, I’m seeing a lot more of that second type of ‘manager’.

Verifying computer output shouldn’t be a novel idea. Wisconsin statutes require clerks to keep an ‘auditable’ paper record of every ballot, not a ‘decorative’ one, so we can safely infer that legislative intent had something to do with checking accuracy. No sensible person doubts the need to check accuracy, whether the output is our property tax bills or our election results. 

Responsible election officials can converse about the practicalities: How many ballots do we have to examine before we can be confident we’ve identified the right winners? How can we do that in the time available? If I innovate, what opposition can I anticipate and who is willing to back me up?

Those managers’ conversation focuses on reasonable problem-solving, not excuses for inaction.

But there are many election officials who, when they catch even a gentle breeze of change, batten down the hatches as if expecting a hurricane.

Late last month, I sent a for-your-information letter to each of the 61 municipal clerks in a county where we’re working on raising awareness of the need for verification and its practical solution.

Because many are sensitive to any hint of criticism, the letter made it clear that we were asking for change in only county procedures, not municipal ones. In describing what we would be asking of the county, I chose the most respectful, non-threatening, benign words possible without misleading them about the fact that we are asking for improvement.

The letter ended with an offer to visit the municipal clerk, at his or her convenience, to talk more about the issue.

Two weeks later, I found one response in my email inbox. After a few comments that revealed significant naiveté about the inherent limitations of the local clerks’ IT security measures, the municipal clerk wrote:

I tire of these subtle (and sometimes blatant) indictments of a system where it is obvious that no one has taken the time to confer with the clerks first. Instead, they pattern advice on what is assumed to be true.  Also, I get the impression someone is prepared to sell us some proprietary software to “fix” a non-problem.

Well, at least he responded. That put him ahead of the other sixty municipal clerks.

The devil on my shoulder wanted to respond by calling the writer’s bluff by pointing out that if he was genuinely willing to confer, he could have accepted my offer to meet rather than complain that we’d never met. 

But I shushed that suggestion. In my actual response, I explained some of the basics of prudent IT security as briefly as I could.

I did not bother to reiterate my offer to meet with him, and I don’t expect to hear back. When someone has boarded up all his mental doors and windows, my guess is that the best thing is to let him calm down in silence until he realizes there won’t be a hurricane.

The disheartening thing is that the people who are open to the breezes of change are not those who are in a position to implement improvements.

Voters and poll workers, to start with, are very interested when they learn that election results could be made safe from electronic miscounts. They know how much they have invested in elections, and how tragic it would be to lose it all to a computer glitch, when such a loss could easily be prevented. 

State and national officials are interested, too. It’s the national authorities’ recommendations that we’re trying to get local elections officials to heed.

Last July, when we gave a public demonstration of an efficient new verification method, attendance at the event included the director of the state Government Accountability Board and his chief elections manager; the director of the state League of Women Voters; and a nationally-renowned, federal-government-advising expert on election technology and computer-science professor from a neighboring state.

From within a 30-mile radius, we had also invited 3 county elections officials, 37 county board supervisors, and 61 municipal clerks. Not one of them showed up. Not one.

Among the I-have-a banana-in-my-ear responses I’ve received from local officials over the past few years:

  • Every professional version of “I’m planning on washing my hair that night.” If an election is coming up within two months, they are too busy preparing to talk. If an election is more than two months away, they will be willing to talk about election administration issues when they are more timely.  The City of Madison municipal clerk begged off attendance at one public meeting by saying she’d be too busy registering voters—at 8:00 PM. (She has a staff of seven.)
  • They’re just so busy with everything else all the time, they cannot even read the invitation. One county board supervisor told me not to expect a response from any other county board supervisor because “We get so many emails we don’t read them.”
  • It’s not my job: “You don’t need to talk to me because I’m (not on that committee; not in the majority; from a neighborhood too affected by Voter ID…).”
  • And, of course, silence.

This cannot go on forever. We will get their attention somehow. For all our sakes, I hope it’s before the next serious electronic miscount gives us cause to shout “I told you so!”

Reality Bites: Election Officials’ Remarks on Voting Machine Reliability

October 15, 2015 at 1:24pm — If some genie were to happen by and offer me three wishes, I fear that before world peace, I’d ask to understand our election officials’ lack of interest in verification, when I’m sure they all have the sense to know why banks audit to make sure their computers credited deposits to the correct accounts. It puzzles me deeply. 

Yesterday. The chairman of Wisconsin’s elections authority, the Government Accountability Board, was testifying before a legislative hearing to defend the agency’s continued existence. One of his Republican executioners confronted him with the fact that the GAB had, for many years, neglected to perform statutorily required post-election audits. How could the Board, the questioner badgered, assure Wisconsin residents that their voting machines had counted correctly if they never did post-election audits?*

GAB Board Chairman Judge Gerald Nichol responded–right out in public, on the record, and apparently without embarrassment, “It is true we did not do the audits, but I don’t find that too worrisome because our staff thoroughly tested the systems before they were approved for use in Wisconsin.” 

And–I’m not making this up–that answer appeared to satisfy his questioner. 

Imagine what Judge Nichol or the legislator would have said if a banker testified, “We never audit, but I don’t find that too worrisome because when we bought those computers, we tested one to make sure it was capable of counting correctly.”

The over-the-top ridiculousness of that statement would be immediately evident to either of them in relation to a bank’s computers, but neither seemed to think it in any way remarkable when the output in question was our election results.

Today. I went to a large-group training where about two dozen municipal clerks were in attendance. I had several opportunities for one-on-one chats, and I used them to feel a few officials out about their level of awareness of voting machine accuracy. I’ve learned to start such conversations with praise for their efforts in double-checking that the machines counted the right number of ballots–which they do quite well.

“I’m conducting sort of an informal survey about clerks’ thoughts on voting machine accuracy,” I would start. “I know you know for a fact that the machines always count the correct number of ballots–you’ve got that nailed down. But what’s your level of trust that the machines counted the right number of votes?”

Most conversations go one of three ways from there. The worst conversations don’t even get started (two tonight, but I’ve gotten this reaction on other occasions). The question explodes some landmine: “Our voting machines are accurate! We test and double test and maintain the very best security. Our election results are always accurate, I’d bet my life on it.” Any follow-up question (e.g., Can you tell me what gives you that confidence?) will be met with only more anger, and the clerk or poll worker will walk away. The Topic Must Not Be Raised.

The second typical response is less emotional, but no more productive: Some election officials will be unable to understand the question no matter how you phrase and rephrase it. It’s as if you are asking “What do you do when the sun comes up in the west?” They mentally rephrase the question into something that makes sense to them and start talking about that. The elections official will describe the process for checking that the machines counted the right number of ballots, and I’ll politely wait until she is done, and then ask the question again more specifically: “Yes, but how often do you think the number of votes–that is, the number of votes for Jones, and the number for Smith–are correct or incorrect?” The elections official will repeat the explanation about verifying the number of ballots, or tell you how the machines reject over-votes, or how they check their addition during the municipal canvass, or some other thing. I’m usually the one that ends these conversation, because I start to feel I’m being mean.

The third most typical response is that they ‘correct’ the question rather than answering it. In a you-should-know-this tone, they will say something like: “The machines cannot miscount,” or “Accuracy is the county canvass job.”  I had two of these conversations tonight. They don’t go anywhere, either. Once this type of election official has decided you are just naive, they switch to an all-talk-no-listen mode. 

Of seven or eight conversations I started tonight, only one unfolded into a genuine exchange of information. The clerk responded to my question with, “I would say I’m pretty close to 100% confident, but it does bother me we don’t know for sure. I’ve never discovered any miscount in the pre-election test–that’s what gives me confidence–but I know that doesn’t guarantee they’ll count right on Election Day.”  Wow. Town of York voters, you’re in sensible hands. 

For the life of me, I cannot understand this mental block. It would be easy to say “People are naive about computers,” but this level of blindness affects their thinking about only voting machines–no other computers. 

Some sort of reflexive emotional-defense denial is a possibility–the thought of incorrect election results is just so horrifying that they cannot permit its presence. Maybe, but most of an elections official’s work is dedicated to preventing various mistakes and frauds. It’s simply not believable to me that they could not already have accepted the idea that something could go wrong.

Different officials probably have different reasons. For example, it seems that those who react immediately with anger, taking offense that you would even ask about electronic miscounts, are at some level aware that they might be certifying inaccurate totals–otherwise, why would they be so defensive?

But the others–I genuinely cannot guess.


* Just for the record, with the audit procedures the Board uses, final Wisconsin election results are unprotected against electronic miscounts whether the Board does the audits or not. Even if they had done them, their procedures are not designed to detect and correct incorrect election outcomes. Among other problems, they use a too-small sample size and have no provision for expansion of the audit beyond a single precinct if a miscount was discovered. The audits are performed after election results are certified as final, so unaudited voting-machine output determines the ‘winners’ regardless.