How calibrated are you?

Calibration is a single, checkable property of your judgment: when you say you're 80% sure, are you right about 80% of the time? It has nothing to do with how smart or how well-informed you are, only with whether your confidence levels mean what they claim.

Most of us assume they do, and the research says otherwise. The gap takes about five minutes to measure.

The ten-question test

For each question below, write down a range — a low and a high — that you're 90% sure contains the true answer. Not a guess at the exact value: a range. You choose how wide it is; that's the entire point. Wide ranges are allowed, "I don't know" is not. If you're 90% sure on each, you should expect to get about nine of ten right.

Don't look anything up; the only person you'd cheat is you.

In what year did Gutenberg print his Bible?
How long is the Nile, in kilometers?
How many bones does an adult human body have?
What is the average distance from Earth to the Moon, in kilometers?
In what year was the first email sent between two computers?
How deep is the deepest known point of the ocean, in meters?
How many member states does the UN have?
How much does an adult blue whale weigh, in tonnes?
At the summit of Everest, at what temperature does water boil, in °C?
In what year was the first powered airplane flight?

Answers at the bottom of the page. Count how many of your ranges contained the truth.

Reading your score

9 or 10: calibrated, or your ranges were so wide they said nothing, which you'll know in your heart. 8: slightly overconfident; within noise for ten questions. 5 to 7: normal, which should bother you. You claimed 90% and delivered 60%. 4 or fewer: your "90% sure" is a coin flip wearing a suit.

If you landed in the middle, you're in the majority. Across decades of replications, most people's 90% ranges capture the truth 40–60% of the time; Alpert and Raiffa ran the same test at 98%, where the truth should fall outside twice in a hundred, and it fell outside about four times in ten. In their Sloan Management Review write-up, Russo and Schoemaker put the same kind of test to more than two thousand managers and executives; most missed far more than they should have, and they were badly overconfident on questions from their own industry too. The misses nearly always come from the same place: ranges set too narrow.

Your misses probably don't feel like overconfidence from the inside; each range felt reasonable when you wrote it, which is why miscalibration goes unnoticed until something scores it. And trivia is the easy case, with no stakes and nothing you'd rather believe. Your calibration on "should I take this job" has all of that working against it too.

Training it

Calibration responds to practice, but only one kind: explicit predictions scored against outcomes, with the score visible afterwards. Lichtenstein and Fischhoff showed decades ago that a few rounds of confidence judgments with hit-rate feedback measurably improve calibration; weather forecasters, who are inside that feedback loop every day, come out well calibrated as a profession.

For deliberate practice, two free tools do the trivia loop well: Calibrate Your Judgment (Clearer Thinking's trainer) and Fatebook, which makes logging real-world predictions nearly frictionless. Forecasting platforms like Metaculus work too, if world events are your kind of practice material. Decision journal apps compares the tools by what they can score.

But trivia calibration and decision calibration aren't the same skill, and the research on training transfer says improvement in one domain generalizes imperfectly to others. If what you actually care about is the quality of your career, money, and life calls, the feedback loop has to run on those, which means writing down a prediction and a confidence number when you decide, and scoring it when reality reports back. That's a decision journal; there's a free template. After twenty scored decisions you'll have the personal version of the test you just took, on the calls that actually cost you something.

More guides in Making better decisions.

Full disclosure: I built Reckon, an iOS app that keeps that loop for you and turns your scored decisions into per-domain calibration curves, so the scoring never becomes the chore that ends the habit. One-time, no subscription. The trivia tools above are free and I recommend them either way.

Answers

~1455 · 2. ~6,650 km · 3. 206 · 4. ~384,400 km · 5. 1971 · 6. ~10,935 m (Challenger Deep) · 7. 193 · 8. ~100–150 tonnes · 9. ~71°C · 10. 1903