Hot on HuffPost:

See More Stories

Two Pollsters Take Their Lumps Based on Election Day Results

2 years ago
  0 Comments Say Something  »
Text Size
Two very different polling organizations have taken their lumps since Election Day now that voters have actually spoken and pre-election forecasts can be judged by the results at the polls. One is the venerable Gallup organization whose name, after 75 years in the business, is synonymous with polling. The other is Rasmussen Reports, founded in 2003, which has made itself known by the sheer volume of national and state surveys it churns out during each election cycle.

Gallup is a traditional pollster which, like the Pew Research Center and most major news organizations, conducts its surveys based on telephone interviews, (including cell phone-only households, which it added in 2008). Rasmussen is what is known as an IVR pollster (for "interactive voice response") which relies on automated calls that ask those being surveyed to give their responses by selecting a number on their keypad. Rasmussen does not reach cell phone-only households.

VotersGallup's headache was that its final "generic" congressional ballot poll predicted a whopping 55 percent to 40 percent advantage for Republicans, a result that surprised many because of the size of the margin compared to similar surveys by other organizations. By comparison, polls by Pew, New York Times/CBS News, Wall Street Journal/NBC News and Washington Post/ABC News had Republicans ahead by margins ranging from four to six points.

True, when it came to winning control of the House, the Republican victory was sweeping, netting a gain of 60 seats, more than the 1994 landslide that returned the GOP to power in that year. But the actual popular-vote margin for the congressional elections was much closer to the predictions of the other major pollsters and ended up being about half of what Gallup forecast.

Writing on the Huffington Post's Pollster.com, political scientist Alan Abramowitz said, "Not only did Gallup miss the actual vote margin by a mile, but their projections about the composition of the midterm electorate were also way off the mark. Based on the exit poll results, it appears that the actual electorate was not nearly as male, old, Republican, or conservative as Gallup's final likely voter sample."

We asked Frank Newport, Gallup's editor-in-chief, about the outcome and he conceded, "It does appear that Gallup's 2010 final likely voter estimate of the House vote did in fact overestimate the Republican share of the vote and underestimate the Democratic vote – although the Republicans obviously won big, as the model predicted."

Newport added:
The extraordinary Republican advantage in voting enthusiasm evident throughout the year may have proved a challenge for Gallup's likely voter model and thus its estimate of the 2010 vote was more Republican than it turned out to be, perhaps due to the very the large Republican advantage in the vote among those who had the highest possible score on Gallup's likely voter scale. It is not unusual for Republicans to lead among those with the highest score on the likely voter scale, but the size of the lead in 2010 was larger than is typical. At any rate, we'll be reviewing in the months ahead.

As for Rasmussen, it took a hit from polling analyst and statistical guru Nate Silver who just came out with a rating of pollster performances on his FiveThirtyEight.com blog, headlined in part: "Rasmussen Polls Were Biased and Inaccurate."

Rasmussen ranked last in accuracy among a list of eight pollsters who released at least 10 surveys of Senate and gubernatorial races in the last three weeks of the campaign, according to Silver.

Silver said:

The 105 polls released in Senate and gubernatorial races by Rasmussen Reports and its subsidiary, Pulse Opinion Research (which was used for polls issued by Fox News), missed the final margin between the candidates by 5.8 points, a considerably higher figure than that achieved by most other pollsters. Some 13 of its polls missed by 10 or more points.

Silver also asserted that Rasmussen's polls consistently overestimating the standings of Republican candidates and criticized the pollster for its methodology, saying:
Rasmussen ... generally conducts all of its interviews during a single, 4-hour window; speaks with the first person it reaches on the phone rather than using a random selection process; does not call cellphones; does not call back respondents whom it misses initially; and uses a computer script rather than live interviewers to conduct its surveys. These are cost-saving measures which contribute to very low response rates and may lead to biased samples.

Rasmussen's own post-mortem -- and it said it is working on a more detailed one -- did not address the kinds of questions, Silver raised. We've asked them for a response and will post it when we get it.

UPDATED: A spokesman for Ramussen said, "We have no comment other than to say that we're quite comfortable letting the facts speak for themselves. "

Follow Poll Watch on Twitter

Visit the Poll Watch Home Page and see all the latest polls in one place

Our New Approach to Comments

In an effort to encourage the same level of civil dialogue among Politics Daily’s readers that we expect of our writers – a “civilogue,” to use the term coined by PD’s Jeffrey Weiss – we are requiring commenters to use their AOL or AIM screen names to submit a comment, and we are reading all comments before publishing them. Personal attacks (on writers, other readers, Nancy Pelosi, George W. Bush, or anyone at all) and comments that are not productive additions to the conversation will not be published, period, to make room for a discussion among those with ideas to kick around. Please read our Help and Feedback section for more info.

Add a Comment

*0 / 3000 Character Maximum Comment Moderation Enabled. Your comment will appear after it is cleared by an editor.

17 Comments

Filter by:
Thomas

That Rasmussen is biased is no surprise. That the FOX network uses their polls as gospel is also no surprise. One tells the lie, the other repeats it.

November 10 2010 at 3:12 PM Report abuse -2 rate up rate down Reply
Dawn

Its the election that counts not the polls before the election and if your only voting based on the poll numbers instead of a candidates stand on the issues your a fool. I think the best predicter for polls is the independent vote not the dem or republican. Its the swing that's the thing..

November 09 2010 at 10:32 PM Report abuse +5 rate up rate down Reply
vobox3343

So what you're telling us is the misleads by the Conservatives actually had some effect.

November 09 2010 at 9:26 PM Report abuse -4 rate up rate down Reply
CHICKENSHIT

Ok folks, from one not in the business of political elections, save for rare specific instances when requested to consult for just a short time period (and only if wholly on my terms). Zogby is it. To obtain their findings well in advance, the general public must pay a relatively modest fee. Note, Zogby usually publishes quite a bit of the 'good' stuff for free; however, when (and if) they do, it is just hours before an election. Following any of these other pollsters is a complete waste of time. If cost is a factor, get together what need be just a small group and split the cost.

November 09 2010 at 7:47 PM Report abuse rate up rate down Reply
andrc657

Everyone knows that FOX NEWS and Rasmussen polls are hugely biased toward republicans. I, and many others, have been blogging this for months. The poor performance by Gallup was more surprising.

November 09 2010 at 3:05 PM Report abuse -3 rate up rate down Reply
knezrad

The chief problem with polling these days is like it was in the original inception, when then overwhelmingly rich and at most middle class people had telephones, now only fairly conservative people still retain the landlines used to call to interview folks...modified now by more lower income people and older people who yet refrain from the mobile mess. The fact is that polls are affected by the Heisenberg uncertainty priciple more than anything else we do, and only in the case of the quadrennial census or the general elections should they be taken with any serious attention to their results, as the pool of respondents and the gravitas of the decision is greater by far than a momentary interview which may or may not be intentionally or not slanted by the phrasing, placement, or progression of the questioning of the poll. Plus, sometimes people lie to them. Freedom of speech includes that right as well. :) Adolf Hitler was a Nazi loser, because he tried to found a government and society based on the concept of the big lie, repeated often enough, that can be represented as truth. The idea that some of our fellow citizens might take this opportunity through sheer lightness of spirits to fib to a pollster is not half so unbelievable as that. NB moderator-The Hitler comment is a historical commentary, not a personal attack.

November 08 2010 at 11:15 PM Report abuse +3 rate up rate down Reply
callandrew

What a funny response from Rasmussen--they will let the "facts" or lack thereof, speak for them. No matter how wrong. Classic.

November 08 2010 at 8:45 PM Report abuse +3 rate up rate down Reply
2 replies to callandrew's comment
bmgrz

callandrew: Here are Rasmussens results for the Senate (actual vs projected) Rasmussen was very very close in virtually every race. Note: Actual results are as of 1:30 p.m. on November 8, 2010 and are subject to change. State Candidates Actual Results Final Rasmussen Results Alabama Richard Shelby (R) 65% 58% Roy Barnes (D) 35% 30% Alaska Joe Miller (R) 34% 35% Scott McAdams (D) 24% 27% Lisa Murkowski 41% 34% Arizona John McCain (R) 59% 52% Rodney Glassman (D) 35% 32% Arkansas John Boozman (R) 58% 55% Blanche Lincoln (D) 37% 36% California Carly Fiorina (R) 43% 46% Barbara Boxer (D) 52% 49% Colorado Ken Buck (R) 47% 48% Michael Bennett (D) 48% 44% Connecticut Linda McMahon (R) 43% 46% Richard Blumenthal (D) 55% 53% Delaware Christine O'Donnell (R) 40% 40% Chris Coons (D) 56% 51% Florida Marco Rubio (R) 49% 50% Kendrick Meek (D) 20% 16% Charlie Crist (I) 30% 30% Georgia Johnny Isakson (R) 58% 59% Michael Thurmond (D) 39% 29% Chuck Donovan (I) 5% Hawaii Cam Cavasso (R) 22% 40% Daniel Inouye (D) 75% 53% Idaho Mike Crapo (R) 71% 63% Tom Sullivan (D) 25% 24% Illinois Mark Kirk (R) 48% 46% Alexi Giannoulias (D) 46% 42% LeAlan Jones (G) 5% Indiana Dan Coats (R) 55% 52% Brad Ellsworth (D) 40% 34% Iowa Chuck Grassley (R) 65% 55% Roxanne Conlin (D) 33% 37% Kansas Jerry Moran (R) 70% 61% Lisa Johnston (D) 26% 28% Kentucky Rand Paul (R) 56% 53% Jack Conway (D) 44% 41% Louisiana David Vitter (R) 57% 54% Charlie Melancon (D) 38% 33% Maryland Eric Wargotz (R) 36% 38% Barbara Mikulski (D) 62% 56% Missouri Roy Blunt (R) 54% 52% Robin Carnahan (D) 41% 43% Nevada Sharron Angle (R) 45% 49% Harry Reid (D) 50% 45% New Hampshire Kelly Ayotte (R) 60% 56% Paul Hodes (D) 37% 41% New York Jay Townsend (R) 33% 31% Chuck Schumer (D) 65% 59% New York Special Joe DioGuardi (R) 36% 33% Kirsten Gillibrand (D) 62% 54% North Carolina Richard Burr (R) 55% 52% Elaine Marshall (D) 43% 38% North Dakota John Hoeven (R) 76% 72% Tracy Potter (D) 22% 25% Ohio Rob Portman (R) 57% 57% Lee Fisher (D) 39% 33% Oklahoma Tom Coburn (R) 71% 68% Jim Rogers (D) 26% 26% Oregon Jim Huffman (R) 40% 42% Ron Wyden (D) 57% 53% Pennsylvania Pat Toomey (R) 51% 50% Joe Sestak (D) 49% 46% South Carolina Jim DeMint (R) 63% 58% Alvin Greene (D) 28% 21% Utah Mike Lee (R) 61% 61% Sam Granato (D) 33% 28% Vermont Len Britton (R) 31% 32% Patrick Leahy (D) 64% 63% Washington Dino Rossi (R) 49% 48% Patty Murray (D) 51% 47% West Virginia John Raese (R) 43% 46% Joe Manchin (D) 54% 50% Wisconsin Ron Johnson (R) 52% 53% Russ Feingold (D) 47% 46% U.S. House of Representatives Results North Dakota Rick Berg (R) 55% 52% Earl Pomeroy (D) 45% 42% South Dakota Kristi Noem (R) 48% 49% Stephanie Herseth-Sandlin (D) 46% 44% Vermont Paul Beaudry (R) 32% 30% Peter Welch (D) 65% 64% See 2010 Governor Results .

November 10 2010 at 12:25 AM Report abuse +2 rate up rate down Reply
Theone36952s

@bmgrz tl; dr. You honestly posted the results for all the Senate races? A link would've been much more effective. Plus, stop justifying Rasmussen's inherent bias.

November 10 2010 at 11:20 AM Report abuse -7 rate up rate down Reply
georgeanderson2

I said this before and I will say it again--the polls will never be accurate from now on, based on the demographics of people in the US. Only older, conservative people seem to have land-lines, and the younger rely solely on cell phones as their primary phone. All polls from now on will be badly skewed until they find a way to track cell phones.

November 08 2010 at 8:43 PM Report abuse +6 rate up rate down Reply
pgbrooke

That Rasmussen is biased is well understood. That said, all polling is loosing its accuracy due to the organizations' unwillingness or inability to adopt current technology. Any pollster using land lines is immeidately suspect in this age.

November 08 2010 at 7:58 PM Report abuse +3 rate up rate down Reply
hey South

The new policy makes good sense to me. One should be able to disagree without being disagreeable. In the 50's, Sen. Dirksen of Ill. and Sen. Russell of GA. would be on opposite sides during the day in the Senate Chambers and have dinner together several evenings a week.

November 08 2010 at 7:02 PM Report abuse +5 rate up rate down Reply

Follow Politics Daily

  • Comics
robert-and-donna-trussell
CHAOS THEORY
Featuring political comics by Robert and Donna TrussellMore>>
  • Woman UP Video
politics daily videos
Weekly Videos
Woman Up, Politics Daily's Online Sunday ShowMore»
politics daily videos
TV Appearances
Showcasing appearances by Politics Daily staff and contributors.More>>