Polling Has Changed -- Continued
Josh Zumbrun at WSJ:At the conceptual level, there are things being done that aren’t being disclosed,” said Kathleen Hall Jamieson, the director of the Annenberg Public Policy Center at the University of Pennsylvania.
Dr. Jamieson was a coordinator of a recent effort by the National Academy of Sciences to recommend how to protect the integrity of survey research. It took particular aim at the phrase “representative sample.”
Their guidance ought to update many stylebooks: “The phrase ‘representative sample’ should not be used without explicit acknowledgment of the underlying assumptions, including weighting and modeling assumptions.” They recommend ignoring polls that don’t disclose that information.
There is a stark dividing line between the two main approaches used to build panels. Some surveys still recruit panelists randomly, such as by plucking them off the U.S. Postal Service’s master list of residential addresses or by using other offline sampling methods such as random phone dialing or text-message outreach to assemble their initial panel. This at least preserves an element of randomness. But sometimes, even the panels aren’t recruited via random outreach, but often via online ads and nonrandom methods. In this case, the surveys are even more reliant on modeling and weighting assumptions to get their respondents to mirror the U.S. population.
“Surveys can be valuable even if they’re not nationally representative, but you have to design them for that purpose and be honest about what you have,” said Arthur Lupia, a University of Michigan political scientist who worked on the recommendations.
