News Story

Survey Says: No Consensus On Best Political Polling Method

'There is a gap between what the Michigan pollsters are getting and what the big national polls are getting'

Voters increasingly sense that political polls often have more to do with influencing public opinion rather than measuring it.

But some still might not know enough about the survey process to judge a poll's credibility.

"There are just so many polls coming out now that it's difficult to really keep track of them," said Inside Michigan Politics Editor Bill Ballenger. "Nobody should get overly excited about the results of just a single poll. I tell people that about the only thing they can do is look at all of the polls, average them out, and use their own judgment."

Predicting who will actually participate in an election is a key factor in polling. To do this pollsters typically use data from previous elections to build an electoral model. This model is a prediction of what percentage of voters in an election will be Democrats, what percent will be Republicans, Independents, etc. The models also attempt to break down the prospective voters by categories such as gender and age.

Using their electoral models pollsters set benchmarks for the various voting groups. After a voter survey is taken, the pollsters adjust (or weight) the survey results to compensate for groups that either surpassed or failed to reach the pre-set benchmarks. These adjustments bring the poll results in line with the electoral model that each particular pollster uses.

If the initial electoral model is flawed, chances are that the poll results will be wrong. There are strong disagreements over which models come closest to predicting who will participate in an election. There also are disagreements over various aspects of polling methods.

For instance, (FMW)B, a relatively new Michigan-based polling firm, refuses to use exit polling data from past elections when it assembles its model.

"We don't use exit-polling for our electoral modeling because we believe that it under represents some voting groups and those who vote by absentee ballot," said Eric Foster of Foster, McCollum White & Associates, a Troy-based political consulting firm. "We believe the way we're doing it is more accurate. But not everyone agrees with us."

In mid-August, (FMW)B raised eyebrows among poll watchers with a poll showing Republican presidential candidate Mitt Romney leading Obama in Michigan 47.68 percent to 43.88 percent. If the poll was accurate, the Obama campaign could be fighting an uphill battle in what is considered a must-win state for his re-election chances

Nate Silver, an East Lansing native who works for The New York Times, cast aspersions on the (FMW)B poll by pointing to another poll that (FMW)B did earlier in Florida. Silver said the (FMW)B Florida poll estimated the turnout for voters aged 18 to 30 at just 3 percent.

Foster took issue with the criticism.

"I've explained this again and again," Foster said. "The 3 percent represented the number of respondents, not the percentage of voters aged 18 to 30 our model predicted. We weren't saying there will be only a 3 percent turnout among that age group.

"I'd also like to know what our Florida poll had to do with our Michigan poll," Foster said. "They criticize our poll but say nothing about some of the other polls that are based on exit polling from 2008 and are over-weighted on the Democratic side. Look, everyone who knows me knows I'm a Democrat, I vote for Democrats. It's no fun being criticized by other Democrats, but I just happen to believe that personal political views shouldn't affect the integrity of a poll."

In spite of the flap, many considered Silver's article informative. It pointed out a clear disparity between polls done in Michigan by polling firms that frequently poll nationally and in multiple states and those that usually focus primarily on Michigan. At the time the article was written the average of the national polls had Obama leading in Michigan by 7.3 points. Meanwhile, the average of the polls done by Michigan-focused firms had Obama up just 1.8 points.

"There is a gap between what the Michigan pollsters are getting and what the big national polls are getting," said Paul King, pollster for Marketing Resource Group (MRG) of Lansing. “When I look at all of these national polls. . . .  almost all of them are done by automated phone calls. One recent poll by a national firm was just of registered voters. That in itself makes the poll results suspect.”

Capitol Confidential has reported previously on the value of doing polls of "registered voters" instead of "likely voters."

"Another thing about all of these polls is that, amazingly, very few of them even try to get at the growing number of people who only use cell phones," King said. "I'm working very hard to try to get as many cell phone numbers as I can."

Foster says the method used by (FMW)B is unique.

"We have an analytical system that takes statistical data from the clerks (county, municipal, etc.) all around the state," Foster said. "Our benchmark is 1990 and we get data from that year and every election year since then. We don't use exit polling data, which by the way, isn't even available for pre-2002 elections.

"I'm not about to bad mouth other polls and pollsters," he said. "We are all out here trying to do our jobs. The approach we use is unusual because it's so time consuming. In fact, it's tedious. But we believe that doing that work on the front end allows us to be more accurate on the back end."

Steve Mitchell, of Mitchell Research in East Lansing, said that unlike (FMW)B, he uses exit polling data, but that doesn't mean he thinks (FMW)B model is necessarily flawed.

"Some pollsters use exit polling data and some don't," Mitchell said. "We look at it to help determine turnout . . . things like what percentage of voters are likely to be women, what percentage will likely be union members and so on.

“Every pollster has their own way of making their model," Mitchell said. "I'm not about to criticize any other pollster. What it really all comes down to is whether they get it right in the end. In my view, Foster has got a pretty good record. He called this year's Michigan presidential primary to within three percentage points."

Michigan Capitol Confidential is the news source produced by the Mackinac Center for Public Policy. Michigan Capitol Confidential reports with a free-market news perspective.

Commentary

Economic Development a Campaign Tool

The Washington Post editorialized recently that President Barack Obama set a goal of 1 million electric and plug-in hybrid vehicles on the road by 2015. Just last year, the Obama administration’s Department of Energy produced a report that said, “The goal is achievable.”

These projections turned out to be wildly incorrect.

The DoE thought General Motors alone would make 120,000 Chevy Volts in 2012. Instead, there are about 21,000 Volts total on the road and GM just announced that it was shutting down parts of production for the vehicle because of low sales. And this despite massive state and federal government subsidies for the car.

Political promises are cheap because they involve a disconnected third party spending money that does not belong to them — it’s always easier to gamble with someone else’s cash. This isn’t unique to Washington, and it infects both political parties.

In Michigan, the best example is the bipartisan disaster known as the Michigan Economic Development Corp. The Mackinac Center and its news publication Michigan Capitol Confidential have documented hundreds of examples where political promises failed to live up to the hype. The MEDC’s own YouTube channel, documenting the “best” examples of its success, is a playlist of cronyism failures. Despite this, the program chugs on and garners support from across the aisle.

Economist Thomas Sowell points out why politicians are poor central planners, yet continue to try: "No one will really understand politics until they understand that politicians are not trying to solve our problems. They are trying to solve their own problems — of which getting elected and re-elected are number one and number two. Whatever is number three is far behind."

So-called economic development programs are one of the easiest ways for politicians to look like they are “doing something” for the economy. And a complacent media that always covers the projections but rarely the failures doesn’t help.

Until these programs are ended and when politicians who claim to support the free market begin to practice what they preach, expect to see more of the same failures.

Michigan Capitol Confidential is the news source produced by the Mackinac Center for Public Policy. Michigan Capitol Confidential reports with a free-market news perspective.