Many posts have discussed the problems of surveying public opinion in the 21st century.
At Strength in Numbers, G. Elliott Morris discusses polling averages:
In our 50+1 generic ballot average, the published Democratic margin across polls read about D+7 a couple of weeks ago and reads D+5 today — a paper loss of two points that reader Tyler was asking about. But take the raw numbers for the parties (Democrats at 48.9% and Republicans at 43.4; a change from 49.1 and 42.4 two weeks ago), and the drop is only from D+6.7 to D+5.49 — a drop of about one point, not two.
Unfortunately, this incident of exaggeration is inherent to a tradeoff we made when deciding how to present the data. The general practice in polling analysis is not to show decimals on differences between percentages, because a number like “D+5.4” implies a precision the polls just don’t offer (see: 2024, 2020, 2016, etc). Or, if you’re going to publish decimals, you should include a margin of error either visually it in the text.
The tradeoff is that a small move in the margin between the parties sometimes looks bigger than it is, which is exactly what caught Tyler’s eye here. A 1-point change became 2.
...
But much of that 1-point decline is likely mechanical — a product of what data is available for the average. Over the last 2 weeks, new polls have tended to come from firms that lean to the right and produce numbers that are typically on the lower end for Democrats — Quantus Insights, McLaughlin & Associates (Trump’s pollster), and Morning Consult are a few examples. The average I publish for 50+1 tries to account for some of this availability bias (see my methodology) but there is only so much you can do if your trend is being bogged down by biased data. The high-quality New York Times/Siena poll that had Democrats up 10 points is now over a month old.
This is a dynamic of poll-aggregation that people often miss or ignore. A smart polling average can remove a lot of noise from polling data, but the tool inherently still wiggles around from sampling error and house effects even when the “truth” of public opinion is stable. A poll of 1,000 people has a margin of error of roughly three points — and you’re blending at best half a dozen of those from the last week, and usually less.
This doesn’t mean polls are useless — they are still the best tool we have for predicting elections, and the only one we have for directly measuring how people feel about public life. But it does mean that averages tend to move around somewhat predictably. So how much movement is worth paying attention to? Let’s boot up the historical polling averages and take a look.
