Probabilistic Thinking

Beyond True and False

Most real-world reasoning is not about certainty — it is about probability. Will it rain tomorrow? Probably (70%). Will this policy reduce crime? Likely (based on similar policies elsewhere). Is this news story accurate? Almost certainly (from AP wire) or possibly (from an anonymous blog).

Bayesian thinking formalizes this. You start with a prior probability (your estimate before seeing evidence), then update based on new evidence. If you think there is a 30% chance a candidate will win, and a new poll shows them leading, you update upward — perhaps to 45%. Each piece of evidence shifts your probability. The strength of the shift depends on how surprising the evidence is: a poll showing a 10-point lead shifts more than a poll showing a 1-point lead.

Calibration: The Key Skill

A 'well-calibrated' thinker is one whose confidence matches reality. When they say they are 90% confident, they are right about 90% of the time. When they say 50%, they are right about half the time. Most people are poorly calibrated — they are overconfident. Studies show that when people say they are 99% certain, they are wrong about 40% of the time.

Calibration matters because uncalibrated confidence leads to bad decisions. If you are overconfident in a medical diagnosis, you skip tests. If you are overconfident in an investment, you bet too much. If you are overconfident in a political prediction, you are blindsided by outcomes.

To improve calibration: practice making explicit probability estimates and tracking your accuracy. Am I 60% confident or 90%? Over time, comparing your estimates to outcomes reveals your biases and sharpens your judgment.