Poll Aggregation Models

Why Averaging Beats Any Single Poll

Individual polls are snapshots with sampling error, methodological quirks, and potential biases. Averaging multiple polls cancels out random errors and reduces the impact of any single poll's bias. A simple average of the last 10 polls in a state is typically more accurate than any individual poll, including the best one.

Sophisticated aggregation models go further. FiveThirtyEight's model, developed by Nate Silver, weights polls by recency (newer polls get more weight), sample size, and pollster quality (based on historical accuracy). It adjusts for known pollster biases: if a firm consistently overestimates Democrats by 2 points, its polls are shifted accordingly. The model also accounts for correlation between states (a shift in Pennsylvania likely means a similar shift in neighboring states) and incorporates non-polling information like economic indicators.

Probabilities, Not Predictions

The most important and most misunderstood feature of aggregation models is that they produce probabilities, not predictions. When FiveThirtyEight said Hillary Clinton had a 71% chance of winning in 2016, it also said Donald Trump had a 29% chance. Events with a 29% probability happen regularly; a baseball player with a .290 batting average gets a hit nearly three times out of ten.

The media and the public consistently misinterpret probabilities as certainties. '71% chance' was heard as 'Clinton will win.' When Trump won, many people concluded the model had failed. It had not: it had given Trump better odds than almost any other forecaster. The failure was in how the probability was communicated and understood.

Worked Example: Why the Average Is Tighter Than the Polls

The statistical reason averaging helps is that independent random errors partly cancel. Suppose five polls of a state report the following support for Candidate A:

The simple average is (48 + 50 + 47 + 49 + 46) / 5 = 48.0%. Now consider the uncertainty. A single poll of about 800 people has a margin of error near +/- 3.5 points. But when you average k independent polls, the standard error of the average scales by roughly 1/sqrt(k). With k = 5, that is 1/sqrt(5) = 0.45, so the effective margin of error on the 48% average shrinks to roughly 3.5 x 0.45 = 1.6 points, less than half that of any single poll.

This is why aggregators can distinguish a 48-46 race from a 49-45 race that individual noisy polls cannot. Two caveats keep this honest. First, the 1/sqrt(k) gain assumes the errors are independent; if every pollster shares the same systematic bias (for example, all undercount non-college voters), averaging cannot cancel it, which is exactly what happened in 2016 and 2020. Second, weighted aggregators do not use a flat average: a fresh 1,000-person A-rated poll counts for far more than a stale 400-person C-rated one. The worked average above is the intuition; the real model is a weighted, bias-adjusted version of the same idea.