Campaign 2004: The Exit Poll Mystery: By BOB BURNETT

Friday November 19, 2004

As I worked getting out the vote in Boulder, Colo. on Nov. 2, I received early exit poll data showing Kerry ahead in key states. Indeed, when the Colorado polls closed, all indications favored the Democrats. What troubles me about the final results is that they greatly varied from the exit polls. 

It’s not like exit polling is an unproven technology. Historically American exit polls have been very accurate; so reliable that international institutions use exit polling to validate election results in emerging democracies such as Georgia and Afghanistan. 

In the 2004 presidential election, two organizations conducted exit polls: the National Election Pool, which merged the facilities of Edison Media Research and Mitofsky International, and Brigham Young University, which conducts surveys in Utah. 

The 2004 BYU exit polls accurately predicted the final Utah outcome; they had Bush at 70.8 percent and Kerry at 26.4 percent where the actual result was Bush 71.1 percent and Kerry 26.4 percent. 

In non-swing states the exit polls conducted by the National Election Poll group closely tracked the final tallies; for example, in Missouri, the exit polls predicted the result as 46 percent Kerry and 54 percent Bush where the final result was 46 percent Kerry and 53 percent Bush. 

The problems with exit polls occurred only in swing states. There were eleven “battleground” states: Colorado, Florida, Iowa, Michigan, Minnesota, Nevada, New Hampshire, New Mexico, Ohio, Pennsylvania, and Wisconsin. Only in Wisconsin did the exit polls come close to the final result. In the other 10 swing states the polls were dramatically off. 

What caught my attention was that the variance was not random, some for Kerry and others for Bush; all the final tallies were significantly better for Bush than the exit polls predicted. The average net differential—predicted difference between Bush and Kerry less the actual difference—was 4.8 percent. (For example, in New Hampshire Kerry was predicted to beat Bush 54.9 percent to 44.1 percent, a difference of 10.8 percent; the actual results were Kerry 50.3 percent, Bush 49 percent, a difference if 1.3 percent; therefore the net differential favored Bush by 9.5 percent.) 

Thus, there were two problems with the swing-state exit polls: they were wildly off the mark—4.8 percent is a huge error for these polls—and they all erred in Bush’s favor. 

When asked to explain these abnormalities, Warren Mitofsky, head of one of the groups conducting exit polling for the National Election Polls, gave two excuses. The first was that the “early numbers” shouldn’t have been released to the media, because they were, in fact, early; i.e., the sample size wasn’t adequate. But, the early polls in many swing states were virtually the same as the late polls. For example, in New Mexico the results of the first poll showed Kerry ahead 50 percent to 48 percent and the results of the last poll showed Kerry still ahead 50 percent to 49 percent. 

On the PBS News Hour Mitofsky stated, “we suspect that the reason [Kerry was ahead in the exit polls] was that Kerry voters were more anxious to participate in our exit polls than the Bush voters.” Of course, this excuse doesn’t explain why the exit polls were inaccurate only in swing states. 

And it ignores the fact that the exit polls are carefully constructed samples weighted by factors such as party affiliation and gender. In other words, it is not the case that pollsters grab the 20th voter coming out of the polls and mark down their vote preferences regardless of what party they belong to; polling protocol dictates that if they have too many Democrats, or women, they don’t take anymore in that particular category until the sample is balanced. 

Steven Freeman, a professor at the University of Pennsylvania, conducted a thorough review of the exit poll results in Florida, Ohio, and Pennsylvania. 

These polls predicted a narrow Bush win in Florida, and a substantial Kerry win in the other two states. But, the actual difference between the predictions and actual results favored Bush in all three states; the average being 6 percent. Using careful statistical methods, Freeman calculated that given the exit polls the likelihood of the final result in each state is outside the 99 percent confidence interval. Taking the results together, Professor Freeman estimated that, “the odds against all three occurring together are 250 million to one.”  

Freeman’s thorough analysis left me between the proverbial “rock and a hard place” with regards to the final election result. I could choose to believe that Bush’s performance in the battleground states was a statistical anomaly, believe that the exit polls failed and these failures systematically benefited the incumbent. 

My other choice was to acknowledge that the Nov. 2 presidential election may have been stolen, the results manipulated to favor Bush. While I don’t want to believe this, I have read enough about Karl Rove to think he is capable of such a coup. 


Bob Burnett is a Berkeley writer and computer scientist best known as one of the executive founders of Cisco Systems