Opinion polls are pervasive. But can we trust them?
(Page 2 of 2)
Sometimes the substance of a question can hinge on just one word. When the word “openly” was inserted after “serving” in each question, support dropped to 58 percent and 44 percent, respectively.
In general, the more detailed a pollster’s question, the more illuminating the answers will be. As the Weekly Standard’s Andrew Ferguson has pointed out, “Ask ‘Would you like a Ferris wheel in your backyard?’ and a shockingly high percentage of Americans might say yes. Complicate the question, however — ‘Would you like a Ferris wheel in your backyard if it tripled your electric bill and bumped off the family dog?’ — and the number would drop.”
For decades polls have showed that a majority of Americans support Roe v. Wade, the 1973 Supreme Court decision legalizing abortion nationally. To many Americans, Roe is synonymous with abortion rights, and that to support even a limited right to abortion is to support Roe.
In 2007, the Ethics and Public Policy Center commissioned a national poll of registered voters that attempted to measure what the public knows about Roe. When respondents were simply asked whether they wanted Roe overturned, a majority (55 percent) said “no,” and only 34 percent said “yes.”
Respondents were then given an explanation of what Roe means — that it prohibits states from limiting abortion in the first six months of pregnancy, and that if Roe were overturned, states could pass laws to legalize abortion. With this knowledge, the share of respondents that opposed reversing Roe dropped seven points, to 48 percent, and the share that supported overturning Roe leaped nine points, to 43 percent.
Not that this settles the question. I know partisans on both sides who would object to the above description of Roe and to its stated implications. And if you think trying to explain abortion is hard, try testing the public’s knowledge of stem cell research, global warming, or campaign finance reform.
Another challenge is deciding whom to poll. Days before the 1936 presidential election, Literary Digest released a poll predicting that Republican Alf Landon would win comfortably. Three days later, his opponent, Democrat Franklin Delano Roosevelt, won in the biggest presidential landslide in more than a century. Landon carried only two states and received 8 electoral votes to FDR’s 523.
The Digest poll included 2.3 million people (nearly two percent of the U.S. population). The problem? Sampling bias. Its sample was huge but hardly random, created by combining telephone and automobile ownership listings. Telephones and cars were amenities available mostly to the rich at the height of the Great Depression. So the Digest ended up polling a disproportionate number of wealthy Americans, who were more likely to support the Republican.
So why did Rasmussen’s polls look better for Republicans ahead of the 2010 election?
In large part it was because of who was sampled. While many pollsters sampled “all adults” or “registered voters,” Rasmussen polled “likely voters,” a population that captured more Republicans, who were more enthusiastic about voting this year.
Every decision a pollster makes will affect the poll’s outcome. For example, most pollsters contact people by phone. But some use pre-recorded telephone inquiries (which are cheaper and allow for larger samples), while others conduct live phone interviews.
Why does it matter? Because respondents tend to be more candid with the computerized questioners — less apt, for instance, to exaggerate how likely they are to vote or to lie about holding an unsavory view.
Here’s my advice: The next time you read a headline about an opinion poll, don’t take it at face value. Dig a little deeper. Examine the statistical methods and think critically about the wording of the poll, its sample size, who was surveyed and how they were contacted.
Pollsters are constantly refining their methods. But one thing sophisticated statistical techniques can never completely account for is the complex and sometimes contradictory mind of the respondent.
As E.B. White once said, “The so-called science of poll-taking is not a science at all but mere necromancy. People are unpredictable by nature, and although you can take a nation’s pulse, you can’t be sure that the nation hasn’t just run up a flight of stairs.”
A man of faith in a godless age is hitting Americans where it hurts.
Mr. and Mrs. American Spectator Reader, let P.J. O’Rourke talk sense to your kids.
In Britain, defending your property can get you life.
The debacle of this president’s administration is both a cause and a symptom of the decline of American values. Unless Congress impeaches him, that decline will go on unchecked. An eminent jurist surveys the damage and assesses the chances for the recovery of our culture.
It won’t take long for conservatives to scratch this presidential wannabe off their 2008 scorecard.
The American Christmas, like the songs that celebrate it, makes room for everybody under the rainbow. Is that why so many people seem to be hostile to it?
Was the President done in by the economy, or by the politics of the economy?