Probabilistic Thinking

Beyond True and False

Most real-world reasoning is not about certainty - it is about probability. Will it rain tomorrow? Probably (70%). Will this policy reduce crime? Likely (based on similar policies elsewhere). Is this news story accurate? Almost certainly (from AP wire) or possibly (from an anonymous blog).

Bayesian thinking formalizes this. You start with a prior probability (your estimate before seeing evidence), then update based on new evidence. If you think there is a 30% chance a candidate will win, and a new poll shows them leading, you update upward - perhaps to 45%. Each piece of evidence shifts your probability. The strength of the shift depends on how surprising the evidence is: a poll showing a 10-point lead shifts more than a poll showing a 1-point lead.

Calibration: The Key Skill

A 'well-calibrated' thinker is one whose confidence matches reality. When they say they are 90% confident, they are right about 90% of the time. When they say 50%, they are right about half the time. Most people are poorly calibrated - they are overconfident. Studies show that when people say they are 99% certain, they are wrong about 40% of the time.

Calibration matters because uncalibrated confidence leads to bad decisions. If you are overconfident in a medical diagnosis, you skip tests. If you are overconfident in an investment, you bet too much. If you are overconfident in a political prediction, you are blindsided by outcomes.

To improve calibration: practice making explicit probability estimates and tracking your accuracy. Am I 60% confident or 90%? Over time, comparing your estimates to outcomes reveals your biases and sharpens your judgment.

Worked Example: A Bayesian Update Step by Step

Suppose a rapid test flags a rare disease. The disease affects 1 in 1,000 people. The test catches 99% of true cases (sensitivity) and wrongly flags 5% of healthy people (false-positive rate). You test positive. What is the probability you actually have the disease?

Intuition screams 'about 99%.' Bayes says otherwise. Imagine 100,000 people:

Of ~5,094 positive tests, only ~99 are real. So P(disease | positive) is about 99 / 5,094, roughly 1.9%. The low base rate (prior) dominates: a very accurate test still yields mostly false alarms when the condition is rare.

Why it matters. The same base-rate logic governs security screening, spam filters, and 'this indicator predicted the last recession.' The update was real - a positive test moved you from 0.1% to 1.9%, a nearly 20-fold increase - but it did not reach certainty. Bayesian thinking keeps both truths in view: the evidence mattered and the conclusion is still probably false. Ignoring the prior is the base-rate fallacy, one of the most common reasoning errors in medicine, law, and politics.