Dem
54
GOP
46
Politico asked PF Global to conduct a poll asking Americans what they think about polls. The results are disheartening. Question 1 was: Which of these statements do you agree with? The results are below, with the last three columns giving the crosstabs based on how the person was planning to vote on Nov. 3:
| Statement | Men | Women | Total | Democrats | Republicans | Undecideds |
| Polls underestimate Democrats | 17% | 19% | 18% | 33% | 7% | 2% |
| Polls are accurate | 26% | 24% | 25% | 28% | 26% | 15% |
| Polls underestimate Republicans | 27% | 19% | 23% | 12% | 45% | 8% |
| Don't know | 30% | 38% | 34% | 28% | 21% | 75% |
Only a quarter of Americans think the polls are nailing it. Interestingly, Democrats think the polls are missing Democrats and Republicans think the polls are missing Republicans. This seems to be a partisan reflex more than anything else.
Question 2: The polls are showing Democrats winning key races. Which of these statements do you agree with? Again, the results:
| Statement | Men | Women | Total | Democrats | Republicans | Undecideds |
| Polls are right. Democrats will win them | 43% | 40% | 41% | 70% | 20% | 15% |
| Polls are wrong. Republicans will win them | 26% | 21% | 23% | 7% | 50% | 9% |
| Don't know | 31% | 39% | 35% | 23% | 30% | 76% |
Again here, there is a strong partisan bias. Democrats are very sure the polls are right. This disagrees with the conventional wisdom that Democrats are wetting their pants because the polls are so good they must be wrong.
Among the people who disagreed with the statement that Democrats will win the key races, here are the reasons:
| Reason | Men | Women | Total | Democrats | Republicans | Undecided |
| Polls are missing Republicans | 59% | 61% | 60% | 52% | 65% | 42% |
| Polls are right but things will change by Election Day | 34% | 31% | 33% | 37% | 30% | 26% |
| Don't know | 7% | 7% | 7% | 11% | 5% | 32% |
The most popular reason was the "shy Republicans" who refuse to be polled. But in previous elections, that effect seems mostly to be true only when Trump is on the ballot. In other years, the polls were pretty good. Still, "once bitten, twice shy," as they say.
The polling industry is well aware of the fact that getting a representative sample to actually answer all the questions is very difficult and has tried many ways to increase that. Mixed-mode polling, where people selected from the actual list of registered voters are sent text messages with links to the poll questions seems to get better responses and is often used. Corrections for possible undersampling of certain demographics is common now, although the hard part is figuring out how big the correction should be.
Sometimes pollsters try to get estimates of demographics by other means. For example, it is possible to get lists of people who are registered owners of a pickup truck and lists of people who subscribe to Guns & Ammo magazine. So it can be known quite accurately what fraction of the population owns a pickup truck and what fraction gets the magazine. Surveys can then ask respondents if they own a pickup truck and if they subscribe to Guns & Ammo. If, for example, it is known that 30% of Americans in some state own a pickup truck and the survey shows only 20% who own a pickup truck, it is a good guess that pickup truck owners are underrepresented in the survey by 50%. That can be corrected for by counting each one as 1.5 people. Which is interesting, because many people buy a pickup truck to compensate for the fact that they are half a man.
More generally, pollsters can get outside data to determine what fraction of the population in a state is a male, middle-aged, high-school-only, white, churchgoing, pickup-truck driving, subscriber to Guns & Ammo, which probably correlates well with being a MAGA Republican. If the number of people in the survey with these characteristics is lower than the expected number, the ones who did answer can be appropriately overweighted. By asking seemingly random questions like: "Which of the following magazines do you subscribe to... ?" the pollster is collecting extremely valuable information. In reality, it is more complicated than this, but hopefully you get the core idea of using external data to estimate who is underrepresented in a survey and then correcting for it.
Another problem with polling that is really inherent is that a poll is a snapshot in time. If public opinion evolves quickly in the final week of a campaign, a poll take Oct 24-25 is going to miss it. For example, polls of the Ohio gubernatorial race taken before the now-infamous ad are probably going to be way off from the election result, but not because the pollster goofed up. They will be off because the voters changed their minds. We also saw this in the Democratic primary for governor of Wisconsin earlier this year.
In any event, the pollsters are about to get a big test today. The story they tell is pretty clear (see above), and if it's substantially off, one side is going to be very disappointed, and the other is going to take it as "more proof" that the polls are useless. (V)