-
Eugenic Abortion 2.0
May 23, 2013 | 68 comments
-
Yoani Sánchez Uncaged
April 12, 2013 | 163 comments
-
Take the Helmets Off
February 4, 2013 | 68 comments
-
Let’s Pay Women Not to Abort
January 23, 2013 | 69 comments
-
The Millennial Divide
January 22, 2013 | 30 comments
Political Hay
Polling’s Station
Opinion polls are pervasive. But can we trust them?
(Page 2 of 2)
Sometimes the substance of a question can hinge on just one word. When the word “openly” was inserted after “serving” in each question, support dropped to 58 percent and 44 percent, respectively.
In general, the more detailed a pollster’s question, the more illuminating the answers will be. As the Weekly Standard’s Andrew Ferguson has pointed out, “Ask ‘Would you like a Ferris wheel in your backyard?’ and a shockingly high percentage of Americans might say yes. Complicate the question, however — ‘Would you like a Ferris wheel in your backyard if it tripled your electric bill and bumped off the family dog?’ — and the number would drop.”
For decades polls have showed that a majority of Americans support Roe v. Wade, the 1973 Supreme Court decision legalizing abortion nationally. To many Americans, Roe is synonymous with abortion rights, and that to support even a limited right to abortion is to support Roe.
In 2007, the Ethics and Public Policy Center commissioned a national poll of registered voters that attempted to measure what the public knows about Roe. When respondents were simply asked whether they wanted Roe overturned, a majority (55 percent) said “no,” and only 34 percent said “yes.”
Respondents were then given an explanation of what Roe means — that it prohibits states from limiting abortion in the first six months of pregnancy, and that if Roe were overturned, states could pass laws to legalize abortion. With this knowledge, the share of respondents that opposed reversing Roe dropped seven points, to 48 percent, and the share that supported overturning Roe leaped nine points, to 43 percent.
Not that this settles the question. I know partisans on both sides who would object to the above description of Roe and to its stated implications. And if you think trying to explain abortion is hard, try testing the public’s knowledge of stem cell research, global warming, or campaign finance reform.
Another challenge is deciding whom to poll. Days before the 1936 presidential election, Literary Digest released a poll predicting that Republican Alf Landon would win comfortably. Three days later, his opponent, Democrat Franklin Delano Roosevelt, won in the biggest presidential landslide in more than a century. Landon carried only two states and received 8 electoral votes to FDR’s 523.
The Digest poll included 2.3 million people (nearly two percent of the U.S. population). The problem? Sampling bias. Its sample was huge but hardly random, created by combining telephone and automobile ownership listings. Telephones and cars were amenities available mostly to the rich at the height of the Great Depression. So the Digest ended up polling a disproportionate number of wealthy Americans, who were more likely to support the Republican.
So why did Rasmussen’s polls look better for Republicans ahead of the 2010 election?
In large part it was because of who was sampled. While many pollsters sampled “all adults” or “registered voters,” Rasmussen polled “likely voters,” a population that captured more Republicans, who were more enthusiastic about voting this year.
Every decision a pollster makes will affect the poll’s outcome. For example, most pollsters contact people by phone. But some use pre-recorded telephone inquiries (which are cheaper and allow for larger samples), while others conduct live phone interviews.
Why does it matter? Because respondents tend to be more candid with the computerized questioners — less apt, for instance, to exaggerate how likely they are to vote or to lie about holding an unsavory view.
Here’s my advice: The next time you read a headline about an opinion poll, don’t take it at face value. Dig a little deeper. Examine the statistical methods and think critically about the wording of the poll, its sample size, who was surveyed and how they were contacted.
Pollsters are constantly refining their methods. But one thing sophisticated statistical techniques can never completely account for is the complex and sometimes contradictory mind of the respondent.
As E.B. White once said, “The so-called science of poll-taking is not a science at all but mere necromancy. People are unpredictable by nature, and although you can take a nation’s pulse, you can’t be sure that the nation hasn’t just run up a flight of stairs.”
About the Author
Daniel Allott is a writer in Washington, D.C.
Letter to the Editor View all comments (16) | Leave a comment
More Articles by Daniel Allott
More Articles From Political Hay
http://spectator.org/archives/2010/12/01/pollings-station
Follow us:
ADVERTISEMENT
SPONSORED LINKS
FLASHBACK TO: 1995
-
Pope Culture
A man of faith in a godless age is hitting Americans where it hurts.
-
Why the GOP Does Not, Like, Totally Suck
Mr. and Mrs. American Spectator Reader, let P.J. O’Rourke talk sense to your kids.
-
Dames at Sea
-
In the Absence of Guns
In Britain, defending your property can get you life.
-
The Metaphysical Martini
-
Counting the Costs of Clintonism
The debacle of this president’s administration is both a cause and a symptom of the decline of American values. Unless Congress impeaches him, that decline will go on unchecked. An eminent jurist surveys the damage and assesses the chances for the recovery of our culture.
-
Chuck McHagel
It won’t take long for conservatives to scratch this presidential wannabe off their 2008 scorecard.
-
The Legacy of Old John and Mother Fresh-Roasted
-
It’s the Best Time of the Year
The American Christmas, like the songs that celebrate it, makes room for everybody under the rainbow. Is that why so many people seem to be hostile to it?
-
The Bush Crack-Up
Was the President done in by the economy, or by the politics of the economy?
Clip of the Day
ADVERTISEMENT
Coal Miner's Son| 12.1.10 @ 7:47AM
"There are three kinds of lies: Lies. Damned Lies, and Statistics"---Mark Twain.
David W| 12.1.10 @ 2:45PM
and three kinds of lers - liers, damn liers, and statisticians. I never believe polls. I've developed surveys as part of work and non-work efforts. With little background I know that the wording and sampling size/group can skew the results one way or the other.
Alan Brooks| 12.1.10 @ 8:47PM
Agreed with Mark Twain.
IMO figures lie, and liars figure; yet that includes GOP pollsters-- everyone is grinding an ax, McCain as well.
And I trust the GOP less than the Dems.
Curly Smith| 12.1.10 @ 8:40AM
It's all in how you ask the question. If you read the post in the sidebar "Cantor and ObamaCare Repeal" you'll find that Cantor presumably supports forcing insurers to cover those with pre-existing conditions. According to Rush, it's because it polls well.
A pollster could the question in two different ways:
Do you think insurers should be forced to cover those with pre-existing conditions? (a large percentage would likely say "YES!!")
Forcing insurers to cover those with pre-existing conditions would raise premiums for all other policyholders. Do you think insurers should be forced to cover those with pre-existing conditions? (a large percentage would likely say "NO!!!!!")
Leaders lead and followers follow. If your life revolves around reading polls then you're not a leader. Why don't we just replace Congress with an online voting system? If "leadership" merely involves giving us what we say we want then why do we need the middleman and his graft-laden system?
BG| 12.1.10 @ 10:56AM
There's a greater problem with "sampling." Polling accuracies are directly related to the randomness of the sample.... But in the political arena, there is no truly random event. The Sample is already heavily skewed with Liberals having the far advantage.
Just think about it.... Conservatives are generally working between 8-5, they are by nature reluctant to participate in a poll of any kind, and are at best passive-aggressive with whatever is being asked of them! They don't want to give any pollster the time of day!
Liberals, on the other hand, love to be the center of attention. They fit right in with the kind of person who will respond to the pollsters, being more than willing to engage with everybody about anything, no matter how personal, seeing it as an outlet for their naricissism.
Further, whereas Conservationves will decline to participate rather than lie or missrepresent themselves, Liberals know that masquerading as a Conservative will likely get their opinion processed, while at the same time upsetting what others see Conservatives as thinking.
In other words, "random sampling" is almost an impossibility to assure. Because of the nature of Liberal personalities. they will always be over-sampled. Having taught graduate-level statistics and sampling for may years, I know this to be a fact! It's the nature of the beast... thought pollsters don't want anybody to know this!
Ned| 12.1.10 @ 12:45PM
That sample bias is one of the things that Rasmussen adjusts for - as he describes it, Dims are over-sampled. By making those adjustments, he reduces the impact of liberal group-think, and improves the accuracy of his polls... which leads directly to his vilification by the left... they see only "reduces the impact of liberal group-think" and call it bias, purposefully ignoring that it "improves the accuracy of his polls"...
BG| 12.1.10 @ 2:11PM
That's true, and that's why he and others are vilified by the Left. Still, the means by which Ramussen et al attempt to compensate is still guess-work at best, and a poor attempt in the least because of their time-tables... as you know, even vocabulary is subject to bias as per the political group-think one belongs.... the questions themselves and their presented order are never really adequeatedly tested nor analysed for content or accuracy as noted by the author. Forced-choice agendas are notorious for giving false positives. And guess what, the Liberal mind excells in that exact test-taking agenda, as it points to a sociopathology of those well practiced in deceit and slight of hand.... Go figure!
Ken Roberts | 12.1.10 @ 11:27AM
No they can not be trusted at all, the people wanting to change America have gotten control of that as well. today I see where 90 % of those polled no type or what kind of people just those that were polled said nothing would hurt the military if don't ask don't tell were to be repealed, but I also happened to see a poll taken of the soldiers and a solid 63% were against doing away with don't ask don't tell . so it depends on which poll one believes. polls are somewhat useless any more . Can you imagine if you are sharing a fox hole with someone that is gay the ribbing that would take place from your fellow soldiers . Being a soldier at one time I can . I have never seen a time such as this where the minority rules, it has got to change and soon.
Ray| 12.1.10 @ 1:45PM
There is one pervasive "problem" with "opinion" polls that can never be overcome, and that is human nature itself. Opinions change over time, even in the short term. A given poll may indicate the general population's "opinion" when the poll was taken (never mind how "accurate" the poll may claim be, that's yet another matter of opinion), but that opinion is constantly changing so it's darn near impossible to determine just what the general opinion is just a few days after the polls was taken unless you take another poll. As a historical device, it's very useful. As an analytical device, it's useless.
I look at it this way: an opinion poll is as historical device (a means of recording historical information), and the data should not as a analytical tool, the "data collected" shouldn't be used in statistical analysis, for the results of statistical analysis is only accurate when the subject of study is static, unchanging.
Yes, we can use statistical analysis to determine, predict, how many peanuts are in a jar, but that's because the size and shape of peanuts don't change much once they're put into that jar. Human opinion changes constantly so we can't use statistical analysis to determine, predict, who will win the next election, or even how "popular" someone or something will be a month from now.
Intelligent Design| 12.1.10 @ 1:52PM
Are you in favor of completely eliminating the federal income tax system, along with 67,000 pages of regulations and the IRS, replacing revenue with a simple national sales tax?
Has any major pollster asked that question?
DodAederen| 12.1.10 @ 3:27PM
For I am involved in mankind.
Therefore, send not to know
For whom the cell polls,
It polls for thee.
Richard| 12.1.10 @ 4:16PM
The poll question I most despise is; "Do you think the country is headed in the right direction?"
There are enough variables in that question to make the answers meaningless.
Just try jotting down on a piece of paper the combinations and permutations of a lefty and a righty comparing what each thinks he is seeing to what actually is happening versus what he wants to see. Suggestion - use a new roll of TP on which to record your findings in portrait, not landscape, format.
Tom Daniels| 12.1.10 @ 9:29PM
Article: "Rasmussen was accurate again this year, with the only major miss its projection that Sharron Angle would defeat Harry Reid in Nevada by four points."
I'm sorry that Mr Allott does not understand the difference between bias and accuracy. Mr Allott, like Rasmussen, are both inaccurate and biased in this claim.
Accuracy has to do with how far off polls are on average. Bias has to do with the average direction of error. Accuracy shows how well the poll was conducted. Bias shows how honest the poll was.
In terms of accuracy, Rasmussen was on average 4% off, which was more than every other major poll.
In terms of bias, they consistently showed Republican bias. An elementary probability theory analysis shows that with at least 99.9% confidence, the data from Rasmussen was slanted towards Republicans. A more complex analysis moves that probability well over 99.9999%. These analyses have been well documented on the web: anyone with a little statistical knowledge can verify them.
As for the claim that Nevada was the only major mistake from Rasmussen, have a look at how Rasmussen did on Hawaii (the largest error ever recorded by any poll), Delaware, West Virginia, Alaska, Ohio, California, and Colorado Senate races. All were erroneous outside the error margins of Rasmussen's polls.