Nonresponse Bias in 2024
In fact, the industry did not solve its problems last year. In 2016, pollsters famously underestimated Donald Trump by about 3.2 points on average. In 2024, after eight years of introspection, they underestimated Trump by … 2.9 points. Many of the most accurate pollsters last year were partisan Republican outfits; many of the least accurate were rigorous university polls run by political scientists.
...
The most basic problem in 2024 was the same as in 2016: nonresponse bias, the name for the error that is introduced by the fact that people who take polls are different from those who don’t.
A pollster can weight their way out of this problem if the difference between those who respond and those who don’t is an observable demographic characteristic, such as age and gender. If the difference is not easily observable, and it’s correlated with how people vote, then the problem becomes extremely difficult to surmount.
Take the fact that Trump voters tend to be, on average, less trusting of institutions and less engaged with politics. Even if you perfectly sample the right proportion of men, the right proportions of each age group and education level, and even the right proportion of past Trump voters, you will still pick up the most engaged and trusting voters within each of those groups—who else would spend 10 minutes filling out a poll?—and such people were less likely to vote for Trump in 2024. So after all that weighting and modeling, you still wind up with an underestimate of Trump. (This probably explains why pollsters did quite well in 2018 and 2022: disengaged voters tend to turn out less during midterm elections.)
