Regression to the Mean

The Most Misunderstood Phenomenon in Statistics

Regression to the mean is the statistical tendency for extreme values to be followed by values closer to the average. If a student scores exceptionally well on one test, they are likely to score somewhat lower on the next — not because they got worse, but because extreme performances include a component of luck that does not repeat.

Francis Galton discovered this phenomenon in the 1880s while studying the heights of parents and children. Very tall parents tended to have children who were tall but shorter than them. Very short parents tended to have children who were short but taller than them. Both groups 'regressed' toward the population average.

Why It Matters: Phantom Causation

Regression to the mean is constantly misinterpreted as evidence that an intervention worked or a change occurred. A city records its worst crime year ever, introduces a new policing strategy, and the next year crime drops. The mayor credits the new strategy — but crime was statistically likely to drop anyway simply by regressing toward the average.

In medicine, patients typically seek treatment when symptoms are at their worst. Any natural fluctuation toward the average will look like the treatment worked. This is why randomized controlled trials are essential — they separate genuine treatment effects from regression to the mean.

The 'Sports Illustrated cover jinx' is pure regression to the mean. Athletes who appear on the cover have just had exceptional performances. Their subsequent performances are likely to be less exceptional — not because of a curse, but because extreme performances are statistically unlikely to repeat.

Worked Example: The 'Sophomore Slump' Is Just Arithmetic

Regression to the mean is the tendency for extreme measurements to be followed by more ordinary ones, simply because part of any extreme result is luck, and luck does not repeat. It is one of the most under-recognised traps in data interpretation.

Take a class of students sitting two equally difficult exams. A score is roughly true ability plus a random component (a lucky guess, a good night's sleep, a question that happened to suit you). Look at the students who scored in the top 10% on the first exam:

The top group's average falls on the second exam and the bottom group's average rises — even though nothing was done to either group and their true abilities are unchanged. The reason is selection on an extreme: the top scorers were partly skilled and partly lucky, and on retest the luck component averages out, pulling them back toward the overall mean. The same logic in reverse lifts the bottom group.

This creates a dangerous illusion of causation. Suppose you scold the students who did worst and praise those who did best. On the next exam the scolded group improves and the praised group declines — 'proof' that criticism works and praise backfires. In reality you would see exactly the same movement if you had done nothing at all. Any time a group is selected because it was extreme — worst-performing hospitals, top mutual funds, athletes on a hot streak — expect some reversion, and never credit an intervention until you have compared it against a proper control group.