The Replication Crisis

When Science Cannot Reproduce Its Own Results

In 2015, the Open Science Collaboration attempted to replicate 100 psychology studies published in top journals. Only 36% of the replications produced statistically significant results matching the original findings. This was not a scandal unique to psychology — similar replication rates have been found in cancer biology (25%), economics (61%), and social sciences (62%).

The implications are staggering: a significant proportion of published scientific findings may be wrong. Policies, medical treatments, and public beliefs based on these findings may rest on shaky foundations. The 'replication crisis' does not mean science is broken, but it does mean that any single study should be treated with more caution than most people and journalists give it.

Why Studies Fail to Replicate

Publication bias. Journals prefer to publish positive, surprising results. Studies that find nothing interesting are rarely published, creating a 'file drawer problem' where negative results disappear. This distorts the published literature toward false positives.

P-hacking. Researchers (often unconsciously) try multiple analyses until they find a statistically significant result. With enough variables and analysis choices, random data can produce 'significant' findings.

Small sample sizes. Studies with few participants detect effects that are often noise. The effects that pass the statistical significance threshold in small studies tend to be the most extreme — and therefore the least likely to replicate.

Perverse incentives. Academic careers depend on publishing frequently and in prestigious journals. This creates pressure to produce novel, significant findings — exactly the kind of findings most likely to be false positives.