Florida 2000 and Washington 2004

There is considerable variance between differing technologies within these comparisons, much of which remains after known after controlling for other factors as much as possible. In presidential elections, paper ballots show residual vote rates nearly a full percent lower than optical scan technologies, and almost 2.6 percent better than Datavote punch card machines. The differences between these are statistically significant. In senatorial and gubernatorial elections paper and optical scan perform comparably and both run about 0.9 to one percent better than Datavote punch-card methods, the differences once again being statistically significant. In this case paper and optical scan methods will do better or worse than each other by median estimate depending on whether the evaluation is on a county basis or total voter basis, but the differences overall are not statistically significant (Caltech/MIT, 2001; Ansolabehere & Stewart, 2005). In some areas such as New Hampshire, manual paper has shown higher tabulation discrepancies than optical scan methods at the township and precinct level in recount situations (Ansolabehere & Reeves, 2004) but it's not clear how this would relate to county and state level manual recounts of optical scan base counts, where the impetus for the recount it directed specifically at recovering spoiled ballots. Comparisons of optical scan to paper and level methods yields error rates ranging from 0.1 to around 1.4 percent of all ballots cast. In these cases, there is no clear signal differentiating the two, but the standard deviation reflected in actual historical data is around 0.4 percent of all ballots cast so the "raw" technology effect variability is at least that much. Much of this difference is due to the variability of paper ballot counts as to no one's surprise these are sensitive to variations in the guidelines and procedures by which they are conducted. As such, the noise inherent in paper methods can be driven down considerably with appropriate procedures and full bipartisan oversight. Given that paper sets the standard for reducing residual vote as a baseline that the best machine technologies match but do not significantly exceed, manual recounts using stringent bipartisan guidelines can improve even on primary paper ballot elections giving the most accurate practical vote counts.

Washington state's 39 counties uses a mix of optical scan, DRE, and punch card technologies for machine counts, the latter being dominated by machines manufactured by Webb Systems. Of these, roughly one-third use punch card methods. With the exception of Thurston and Whatcom counties, all are predominately rural. Another 5 use a mix of optical scan and DRE technologies, and the remainder use optical scan systems that are obtained from 3 or 4 manufacturers. Thus, almost half of all Washington counties make at least partial use of systems that lag behind manual paper ballot and optical scan methods and a third use punch card methods that lag considerably behind both. By population, in the fall 2004 election approximately 20 percent of all ballots cast used punch cards and nearly all of the reminder used optical scan (WA Sec. of State, 2004; 2004b; 2004c; 2005b). King County, where most of the Gregoire/Rossi recount attention was focused, uses Global Accuvote optical scan systems, which typically show machine driven residual vote rates of around 0.6 to 0.7 percent nationally as determined from presidential elections (0.3 percent undervote plus 0.3 to 0.4 percent overvote) in good agreement with middle to upper end residual vote rates for optical scan systems in general (Brady, 2000; Ansolabehere & Stewart, 2005; Wolter et al., 2003; Caltech/MIT, 2001). Actual spoiled ballot rates will generally differ because they include both technology and non-technology related factors. These will be closer to observed residual vote rates by county and technology corrected for intentional abstention. Data from exit polls and post-election surveys indicates that roughly 0.5 percent of voters will intentionally abstain from a presidential choice (Ansolabehere & Stewart, 2005). From 1988 to the present senatorial and gubernatorial residual votes attributed to unfixed technology (e.g. technology plus all contributing factors related even indirectly to technology implementation) are around 3 percent and 2.1 percent respectively for optical scan technologies, and 4.4 percent and 3.3 percent for punch cards. For presidential races the corresponding figures are 1.6 percent and 2.5 percent respectively (Ansolabehere & Stewart, 2005).

Federal Election Day Survey results for the fall 2004 election in Washington show residual vote totals of 21,024 and 61,306 for the presidential and senatorial races after write-ins have been accounted for (WA Sec. of State, 2005). Undervote and overvote totals for the presidential and senatorial are included, but these figures aren't available for the gubernatorial race. However residual rates for all 3 races prior to correction for write-ins are available from the Secretary of State (WA Sec. of State, 2004b; 2004e; 2004f; 2004g). These show totals of 24,415, 64,848, and 75158 for the presidential, senatorial, and gubernatorial elections respectively yielding residual vote rates of 0.85 percent, 2.25 percent, and 2.61 percent. These figures are well within the historical range for these rates over the nation as a whole. Though intentional abstentions in the senatorial and gubernatorial races are almost certainly higher than those for the presidential one by proportion, the unintended residual vote is likely to be similar for all three and if anything, worse for the first two (Ansolabehere & Stewart, 2005). This implies that the unintended gubernatorial rate corrected for write-ins can be inferred from the FES presidential and senatorial figures for undervotes and overvotes. Assuming the historical intentional abstention rate of 0.5 percent for presidential election yields an unintentional residual vote rate of around 0.23 percent for the state as a whole. Similar calculations for the state's optical scan counties yield a lower end estimate of 0.11 percent (a more complete breakdown will be given shortly). These figures agree well with the lower end of historical estimates (Ansolabehere & Stewart, 2005). Considering that last fall's local and national elections were hotly contested and likely to have had far lower than average intentional abstentions the agreement is exceptional. Furthermore, the original senatorial residual rate was 0.35 percent lower than the gubernatorial rate making these figures a conservative indicator of the gubernatorial unintended residual rate (WA Sec. of State, 2004b; 2005).

Based on these estimates, it's likely that last fall Washington's overall fixed rate unintended ballot spoilage was at least 6,600. This figure must be taken as a low-end estimate. Ballot spoilage not obvious to machines (e.g. readable ballots that were mismarked due to voter confusion, or double counted ballots for instance) are not included. With these factors accounted for, if Washington's Global Accuvote systems performed anywhere near the national average for such systems, another 0.2 to 0.3 percent spoilage beyond this is possible, which would bring the total to between 12,000 and 16,000 spoiled ballots. All of this assumes historical rates of intentional abstention that are unlikely to be characteristic of last year's heightened partisan tensions. If these reduced the intentional abstention rate by only 0.1 percent there would be an additional 2,800 lost votes.

Compare these figures with those the WSRP offered as "proof" of fraud for over 6 months. Early in January 2005 Chris Vance was touting that 8,500 votes were unaccounted for in voter registrations if King, Snohomish, Pierce, Mason, and Kitsap Counties. Of these, 3,539 were said to be in King County. Within 48 hours it was discovered that this figure had been based on preliminary voter/vote checks that were not to be completed for several days (McGann, 2005; Postman, 2005d). The verified count went down dramatically in following weeks following more thorough investigations. For the next 5 months claims and counter-claims were made. At times the figures being cited changed almost on a daily basis. Vance and the WSRP alleged anywhere from the low to mid-thousands of ballot discrepancies statewide with Democrats arguing that the real figures were much lower. Most of the criticism focused on King County where the manual recount made the largest contributions to the change in margin and canvassing problems received the most public scrutiny. By the election challenge trial date the total had been whittled down to 1,091 questionable votes in King County including Election Day and absentee ballots (Postman, 2005). These and the few hundred mishandled ballots that had been verified were the foundation of the WSRP's claim of election fraud. According to Chris Vance,

"Nobody has confessed to vote fraud... But if the books don't balance and you can't figure out why, you have to assume fraud took place."

(Postman, 2005)

No, you do not. Fraud is a possibility, if the discrepancy in the books is larger than the background noise in the vote tallying process, which neither Vance nor Rossi supporters ever investigated properly. A check of the manual recount by county reveals a total of 876,452 votes for King County across all candidates (WA Sec. of State, 2004c). The verified vote discrepancy of 1,091 in this count represents 0.12 percent of all ballots cast in King County. This is within 0.01 percent of our low-end unintended residual rate for Global Accuvote optical scan systems and less than one fifth of the corresponding national average (Brady, 2000). The total statewide gubernatorial residual vote was sixty to seventy times larger and at best, the statewide unintended portion of this was six to seven times as large. Even if Vance's original discrepancy of 3,539 had stood up, that figure amounts to 0.4 percent of all ballots cast, which is less than one standard deviation for Global Accuvote systems and less than half of the 2-sigma confidence interval on which a statistical estimate of 95 percent confidence would be based.

So, after failing to produce a single independently verified instance of fraud, Vance and the WSRP based their entire case for it based on a ballot discrepancy that was nearly an order of magnitude smaller than statewide technology driven ballot spoilage rates alone. And that before human error of any kind is even be considered, much less negligence or fraud. This is no different than accusing an ophthalmologist of putting a four-letter word in the bottom line of the eye-chart when we can't read a thing below the second line. It's little wonder that Judge Bridges threw the election challenge out.

It's revealing to compare estimates of Washington's technology error rates with the corresponding t-test analysis done earlier and comparable figures from across the nation. The t-test was based on the first order standard error of a gaussian distribution for two proportioned dependent variables--that is, the standard error of the proportion. The denominator of equation 1 gives the pooled standard error for both candidates under those assumptions. For Washington's machine and manual recounts, this works out to about 0.04 percent. Comparable figures are reached for Florida 2000. This is less than one third of our low-end optical scan spoilage estimate, and one sixth of the corresponding statewide rate indicating that if anything, our original t-test was conservative.

How does this compare with other elections across the nation? In August of 2001 the National Commission on Federal Election reform (NCFER) published a full review of the nation's electoral process including recommendations for improvements (NCFER, 2001). Among other things the Commission examined residual vote rates from the year 2000 presidential election for the nation's 40 most populous counties including Washington's King County. Page 55 of that report ranks these by their rates. There, we see that King County had a residual vote rate of 0.7 percent placing it among the top five nationwide. The best rate achieved was 0.3 percent. In fall of 2004 under the leadership of Ron Simms King lowered it's residual rate to 0.52 percent including write-ins. and 0.39 percent with write-ins removed. For comparison, residual vote rates by state averaged to about 1.1 percent for the year 2004 presidential election and the corresponding optical scan county rates ran around 0.7 percent (Stewart, 2005; Kimbal, 2004; Kimball & Kropf, 2005). Thus, King County has one of the best residual vote rates in the nation, consistently performing better than the national average for optical scan counties alone much less all counties.

However, this was of little value to the county's critics. In the wake of the 2004 election Washington State's media outlets were filled with passionate rhetoric about the dismal failure of the county's election system. County Executive Ron Sims and Elections Director Dean Logan became the targets of months worth of blistering criticism including calls for independent audits of the county's Elections Dept. and even the resignations of both men. When shown then current figures suggesting the county had a vote counting accuracy rate of 99.8 percent Chris Vance said that if a bank with $50 million in assets was 99.8 percent accurate, "$100,000 would be missing and someone would probably go to jail" (Ervin, 2005d). Really? For a family of four with a take-home income of $3000 per month, this amounts to $6. That’s barely enough to buy a tall latte and a scone (in Seattle at least). Banks routinely make errors of that magnitude, which is why most people keep check registers and balance their checking accounts at the end of the month. Vance makes the error sound larger than life by using a figure nearly 8 times larger than the number of ballots that were even cast in the election.

None of this is intended to minimize the problems in last fall's election. No lost vote is acceptable and there is most certainly room for improvement at all levels. But criticisms need to be kept in perspective. While King County can, and should, make improvements to its electoral process, accusations that its fall 2004 performance was an abysmal, negligence riddled failure are not supported by national historical data. Those who are angry at the outcome of an election do the public and their elected leaders a great disservice when they allow passion rather than reliable data to fuel discussions like these.

The Sharkansky Method

At least one serious attempt was made by Rossi supporters to prove that his original victory margins were meaningful using t-test methods. In early December, prior to the hand recount Seattle area conservative blogger Stefan Sharkansky (www.soundpolitics.com) ran his own t-test analysis of the base count and machine recount (Sharkansky, 2004). His investigation was motivated by an earlier commentary by another Seattle area political commentator, David Goldstein of www.horsesass.org (Goldstein, 2004). Goldstein had given a brief overview of two papers from the Caltech/MIT Voting Project that examined tabulation error and residual votes attributable to voting technologies (Ansolabehere & Reeves, 2004; Ansolabehere & Stewart, 2005). He applied the results of these studies to Washington's 2004 gubernatorial base count and machine recounts from which he concluded that,

1. The residual vote rate is the primary statistical measure of the performance and accuracy of voting technologies, and was likely to have been around 1 to 2 percent.
2. The tabulation invalidation rate (i.e. the average discrepancy between successive ballot counts in a given election) is on average 0.56 percent for optical scan technologies.
3. A 0.5 percent tabulation invalidation rate in last fall's gubernatorial election, which had over 2.8 million ballots cast, amounts to 14,000 erroneous votes.
4. The results of the both machine counts of that election were statistical ties.

Sharkansky excoriated each of these claims which he refers to as "horse product." He begins his discussion by informing us that,

With a hand recount looming in our historically close gubernatorial election, there has been much debate over the relative accuracy of hand counts versus machine counts, and the error rate of vote counting technologies in general… most of it uninformed. And now thanks to David, we have even more uninformed debate.

He discusses a couple of research papers, which he apparently read, but didn't understand very well; e.g. 'Using Recounts to Measure the Accuracy of Vote Tabulations: Evidence from New Hampshire Elections 1946-2002'....

I do agree with David that our current voting system is prone to inaccuracies, and that we're not going to emerge from the hand recount with confidence that we measured the will of the voters with ball-bearing precision. I hope after this whole mess we can actually work together for meaningful election reform. But the numbers he's throwing around for error rates and "erroneous ballots" are wildly off the mark, and we are not in a 'statistical tie'. Dino Rossi's TWO victories are exactly that. Victories."

(Sharkansky, 2004)

From here he critiques each statement with his own examination of the residual vote and tabulation invalidation from the Gregoire/Rossi machine counts and a t-test analysis which he claims proves that Rossi's "TWO victories" (his emphasis) were anything but statistically insignificant. He begins by deriving his own estimates of the state residual vote rate, and the tabulation invalidation rate (e.g. the discrepancy between the number of total ballots cast for each vote count) using information that was current at the time. Beginning with residual votes, Sharkansky tells us that,

“The 'Residual Rate' (blank and otherwise disqualified ballots) in Washington was far less than 1% this year. Furthermore, all indications are that the vast majority of blank ballots were really intended to be left blank. If you look at the Presidential race, you'll see that a total of 2,883,499 votes were cast and 2,859,084 votes were counted, so there were 24,415 residuals, or 0.85%. But the SoS page doesn't break out write-in votes and they're included with the other residuals. I don't have ready access to write-in numbers from all counties, but I do have those numbers for King County. The SoS page imputes 4,704 residuals for King, but the "e-Canvas" reports 1,194 write-ins, so the real residual rate in King is only 0.39%. That's more or less equal to the Libertarian vote and about half the Nader vote. That doesn't seem to be an unreasonable number of people who would simply chose not to vote for any of the presidential candidates. Some of those residuals may be unintentionally spoiled ballots. But in the King County gubernatorial recount, the canvassing board managed to convert exactly 717 initial residual ballots into non-residuals, out of 898,238 ballots tallied in the first count. That is only 0.08% of ballots that were plausibly miscast such that there is some reasonable claim that the voter filled out the ballot improperly, but well enough to leave marks from which discernable intent can be inferred."

(Sharkansky, 2004)

With most residual votes relegated to deliberate choice, he moves on to actual vote counting errors. Taking his cue from Goldstein's comments, he informs us that,

"The "Tabulation Error Rate" (the difference between the outcomes of the first count and the recount) in the governor's race was nowhere near 0.56%. It was 0.0040% when looking at the entire state, and even taking the weighted average of the (absolute values) of the counties' errors it is still only 0.0046%. [copy this table into Excel and do the math] This result is so far off the mark of the cited paper (7 standard deviations), that the paper's analysis doesn't seem to have any relevance to the systems and processes we use here in WA state....

There is absolutely no basis for screaming that there were '14,000 erroneous votes!' [David's exclamation mark]. First of all, the so-called tabulation error rate does not give the number of erroneous votes, it only gives the discrepancy between two counts. The true number of erroneously counted votes would, on average, be half of the discrepancy. Second, the number is based on a presumed tabulation error rate (0.5%) that is 125 times larger than what we actually experienced. Third, much of the actual discrepancy between the two counts was explained by the discovery of hundreds of new ballots around the state, and not by discrepancies between different methods of reading a controlled sample of ballots."

(Sharkansky, 2004)

He then proceeds directly to his t-test analysis,