Evaluating Evidence

The Hierarchy of Evidence

Not all evidence is created equal. In science and medicine, evidence is ranked by how reliably it establishes causal relationships. At the top are systematic reviews and meta-analyses that combine results from multiple studies. Next come randomized controlled trials. Below those are observational studies, then case reports, and at the bottom, expert opinion and anecdote.

This hierarchy applies beyond medicine. In policy debates, a well-designed study of a program's effects across multiple cities is stronger evidence than a single success story. A statistic from a government statistical agency is more reliable than a number cited by an advocacy group. A pattern that holds across different countries and time periods is more convincing than a single data point.

Questions to Ask About Any Study

Who funded it? Industry-funded studies are not automatically wrong, but they consistently show a funding effect — studies funded by drug companies are more likely to find positive results for the funder's drug. Always check the disclosure statement.

How large was the sample? Small studies (under 100 participants) are more likely to produce extreme results by chance. Be skeptical of dramatic findings from small samples.

Was it peer-reviewed? Peer review is imperfect but provides a baseline quality check. Preprints (not yet peer-reviewed) should be treated as preliminary.

Has it been replicated? A single study, no matter how well-designed, is a starting point. The replication crisis across psychology, medicine, and social science has shown that many published findings do not hold up when other researchers repeat the experiment.