Weighting on Politics: Recalled vote, party ID and the price of stability

In brief

Weighting polls on party identity, or on how people recall voting last time, narrowed the gaps between 2024 polls, and deserves some credit for that. But the gain is tangled up with the other things the firms using those methods were already doing, so nobody can say the weighting itself did it. The method also steers a poll toward the country as it voted four years earlier, and 2024's new voters did not match that. Treat tighter agreement between polls as agreement, not as repair.

How to use this

When you meet a poll weighted on party or on recalled vote, ask what target it was matched to and when that target was measured. Prefer recall collected soon after the previous election to recall gathered years later, and check whether the adjustment allows for new voters or assumes the last electorate. Read a tight cluster of polls as shared method rather than as proof the samples got closer to voters, and ask what the firms share before taking reassurance from it. Never choose a weighting target because it produces the number you or a sponsor wanted.

What the story is about

The sharpest case from 2024 came from Iowa. J. Ann Selzer's final Des Moines Register and Mediacom Iowa Poll, published on 3 November 2024, interviewed 808 likely voters, with a stated margin of error of 3.4 points, and weighted them on age, sex and congressional district. It deliberately left out party, and left out how people said they had voted before. The poll put Harris ahead 47 to 44. Trump carried Iowa by 13. On the margin, that is an error of 16 points.

Three University of Iowa political scientists then read Selzer's own technical review of the poll, which contained a reweighted table based on the same interviews on how those people recalled voting in 2020. DeRagon, Osborn and Lewis-Beck (2024) report that the reweighted table "yields a strong forecast of a Trump victory, with a plus 6 percentage point lead over Harris". The same 808 people moved nine points toward Trump, and the poll was still seven points short of the result. Selzer's reason for leaving past vote out is longstanding: she "has not found voters' recollections of their past votes to be highly reliable".

Across the 2024 cycle as a whole, the picture looks better. The American Association for Public Opinion Research's task force found that "Weighting on partisanship was associated with modestly improved accuracy" (American Association for Public Opinion Research, 2025). It also found that "No single methodological choice guaranteed more accurate results". Its own figure caption cuts both ways: weighting on self-reported partisanship sometimes beat other methods, while weighting on party registration drawn from official voter records sometimes did worse. The polls were also unusually tightly clustered, and the task force puts that down to real convergence in field practices rather than firms copying one another, an argument about clustering that herding and dispersion covers properly. What the task force will not claim is that party weighting caused the gain: "it remains unclear whether these gains reflect the effectiveness of party weighting itself or the broader set of additional practices adopted by firms who use it".

Pew Research Center changed its mind. It has weighted its American Trends Panel on party affiliation since 2014, and in June 2025 it added past vote: "As of June 2025, we are adjusting our surveys to match the results of the 2024 presidential election, too" (Kennedy et al., 2025). Pew gave three reasons. Its polls had underestimated Trump support in 2024 even though they already weighted on party. States had finished updating official records confirming who voted, which supplied a fresh target. And in May 2025 a wave of research, which the post does not name, indicated that weighting on past vote either helps accuracy or does not hurt it. Pew put its own number on the change: "The overall effect of this additional weighting adjustment is less than 1 percentage point, on average".

A recalled past vote is a memory, and memories move. Pennay et al. (2023) tested this on an Australian online panel built to be representative, a month before the 2019 federal election, weighting the same sample seven ways. Weighted on demographics alone, the average distance between the poll's estimate and the parties' shares of first-choice votes (the average size of the miss, ignoring direction) was 2.58 points. Add recall of the 2016 vote asked three months after that election, and error fell to 1.41. Add recall of the same vote asked three years later, and error rose to 2.95, worse than demographics alone. The authors conclude that past vote is "capable of reducing bias in their election forecasts under the right circumstances". The risk is stated plainly by AAPOR: "Self-reported vote is subject to recall error, social desirability bias, and partisan misreporting" (American Association for Public Opinion Research, 2025). The drift has a direction. Van Elsas et al. (2014) found in panel data that people's accounts of their past vote shift toward the party they currently prefer. Then came 2024, when surveys signalled that people who had not voted in 2020 leaned Republican yet still underestimated their share of the electorate. Weighting toward the last electorate assumes the country still looks like it did.

The evidence base under all of this is thin. The one real-data test of weighting on past vote is Pennay et al. (2023): Australian, a single election, cited once. The numbers where a real poll changes the answer come from reports that are not peer-reviewed, and each is attributed to whoever computed it.

One October 2024 exercise shows how much room the choices leave. Josh Clinton took a single poll of 1,718 people and built twelve defensible estimates from it (Clinton, 2024). He varied the raw figures, three sets of demographic targets drawn from past electorates, two benchmarks, meaning assumed party balances matched against an outside source, two assumptions about how many new voters to add when weighting on past vote, and a likely-voter layer on top. The twelve landed between Harris +0.9 and Harris +9.0, a spread of nearly eight points. Clinton's own summary is that "Simple and defensible decisions by pollsters can drastically change the reported margin between Harris and Trump". Two things follow. Every one of the twelve had Harris ahead nationally, and she lost the national popular vote. Weighting choices explain the gaps between polls, not which way the whole set leaned, and that remaining lean is what partisan nonresponse covers.

In 2024, weighting on party and on recalled past vote went together with a narrower spread between polls. The improvement is entangled with what the firms that chose those methods were already doing, and it rests on an electorate that had moved. A poll steered toward the country as it was four years ago buys agreement between polls, and gives up some ability to notice that the country has changed. Both things held at once in 2024: the polls agreed more closely with each other, and the assumption underneath that agreement was not one the election honoured.

So what

Political weighting bought the 2024 polls something small and real: closer agreement with each other. It did not repair them. The method rests on a shaky memory of the last election, and the evidence for it covers one country and one cycle. Treat the tightening as a gain in agreement, not a repair. When a poll's party balance is matched to a past election, the number that comes out has been steered toward an assumption about who votes, and that assumption can be wrong in ways the poll itself cannot show.

For political parties

Parties should read the tighter 2024 cluster as a sign that the industry converged on shared method, not as proof that samples got closer to the voters. From outside, those two look alike and mean different things. A tighter cluster says the firms are doing similar things; it does not say those things are right. Parties should also expect the remaining lean of the polls, the part that pushed estimates toward one side across the board, to sit outside anything weighting can fix. That shapes how a party should use a poll. Choosing a weighting target because it produces the number the sponsor already wanted turns the poll into a mirror. The test is whether the choice serves the decision or the preferred conclusion.

For government

Governments should treat weighting on politics as an evolving, self-reported craft practice, not as a settled standard worth mandating. The case for it rests on a single election cycle, and no peer-reviewed evaluation of its 2024 use exists. That is not a reason to dismiss the method. It is a reason to move slowly. A rulebook written around today's practice locks in an assumption about the electorate that the next election may overturn, and the cost of that is a government reading a number that flatters it instead of warning it.

References

American Association for Public Opinion Research (2025) Task Force on 2024 Pre-Election Polling: an evaluation of the 2024 general election polls. Chaired by J. Pasek. Alexandria, VA: AAPOR, 29 October. Available at: https://aapor.org/announcements/2024-pre-election-polling-report/ (Accessed: 9 September 2026).

Clinton, J. (2024) 'Poll results depend on pollster choices as much as voters' decisions', Good Authority, 28 October. Available at: https://goodauthority.org/news/election-poll-vote2024-data-pollster-choices-weighting/ (Accessed: 9 September 2026).

DeRagon, S.J., Osborn, T. and Lewis-Beck, M.S. (2024) 'The 2024 Iowa poll for president: a cautionary tale', Sabato's Crystal Ball, 12 December. Available at: https://centerforpolitics.org/crystalball/the-2024-iowa-poll-for-president-a-cautionary-tale/ (Accessed: 9 September 2026).

Kennedy, C. et al. (2025) 'Why and how we're weighting surveys for past presidential vote', Decoded, Pew Research Center, 23 July. Available at: https://www.pewresearch.org/decoded/2025/07/23/why-and-how-were-weighting-surveys-for-past-presidential-vote/ (Accessed: 9 September 2026).

Pennay, D., Misson, S., Neiger, D. and Lavrakas, P.J. (2023) 'How weighting by past vote can improve estimates of voting intentions', Survey Practice, 16(1), pp. 1-14. Available at: https://doi.org/10.29115/sp-2023-0001 (Accessed: 9 September 2026).

Van Elsas, E.J. et al. (2014) 'Vote recall: a panel study on the mechanisms that explain vote recall inconsistency', International Journal of Public Opinion Research, 26(1), pp. 18-40. Available at: https://doi.org/10.1093/ijpor/edt031 (Accessed: 9 September 2026).