Do decision journals actually work?
If you go looking for evidence that decision journals work, you'll quickly meet a statistic: journaling your decisions improves forecasting accuracy by around 19%, per a study in Behavioral Science & Policy. I went looking for that study, because I wanted to cite it. I couldn't find it. Not in the journal, not in the literature, not anywhere except the blog posts citing each other. Maybe it exists somewhere I couldn't reach, but a number you can't trace to a primary source is a number you shouldn't repeat, so let's set it aside.
What I can offer instead is the honest version: to my knowledge there is no large randomized trial of decision journaling as a complete practice. What exists is strong evidence for each component of the practice: the mechanism it fixes and the mechanism it borrows. I find the chain persuasive; you can judge for yourself.
The problem is documented: you can't trust your memory of your own judgment
In 1972, before Nixon's trips to China and the USSR, Fischhoff and Beyth asked people to predict specific outcomes: would Nixon meet Mao, would the US recognize China. Months later, they asked the same people to recall their predictions. The recalled predictions had migrated toward what actually happened, and subjects were sure the migrated versions were what they'd said. Hindsight bias is one of the most replicated findings in judgment research, not a niche lab effect.
The implication for learning is brutal: any attempt to improve your judgment by remembering how past calls went is corrupted at the source. The feedback loop runs through an archive that rewrites itself. A written prediction, timestamped before the outcome, is the cheapest known fix: it's the one version of your judgment that can't be rewritten.
The second problem is documented too: your confidence is miscalibrated
Ask people for ranges they're 90% sure contain the true answer — the length of the Nile, a company's revenue — and the true value should escape the range one time in ten. In the classic Alpert and Raiffa experiments and in decades of replications with students, managers, and physicians, it escapes four to six times in ten. Russo and Schoemaker ran versions of this with over two thousand professionals; most were dramatically overconfident, in their own fields as much as outside them.
So the raw material a decision journal collects — "how sure was I, versus how did it go" — is measuring something that is, for most of us, broken and invisible without measurement.
Calibration improves with scored feedback
The encouraging part. Lichtenstein and Fischhoff showed in 1980 that calibration training works: give people rounds of confidence judgments followed by feedback on their actual hit rates, and their stated confidence starts tracking reality, with almost all of the improvement arriving after the first feedback session. The existence proof outside the lab is weather forecasters: Murphy and Winkler found US forecasters' probability-of-rain calls almost perfectly calibrated. Meteorologists aren't a different species; they make explicit probabilistic predictions daily and get scored against outcomes, forever, and calibration is what that loop produces.
A decision journal is that loop, pointed at your own life: explicit probability, recorded outcome, running score.
Practice with scoring improves forecasting itself
The Good Judgment Project — the multi-year forecasting tournament behind Tetlock's Superforecasting — tested this at scale. In Mellers and colleagues' studies, a short training module in probabilistic reasoning improved forecasting accuracy by roughly 10% across a tournament year, and the effect held up in follow-up work on training and practice. The tournament's broader finding matters just as much: the forecasters who kept improving were the ones making granular probability estimates, updating them as information arrived, and running post-mortems on their misses. That triplet is a decision journal's Parts 1 through 3 under a different name.
What this doesn't prove
Candor requires the list. The forecasting results come from geopolitical tournaments, not personal decisions; transfer to "should I take this job" is plausible but assumed, not demonstrated. Calibration training generalizes imperfectly across domains. And journaling has a selection problem in the wild: the people who keep one are already the people who care about their judgment. If someone runs a proper trial of the full practice someday, I'll link it here whichever way it comes out.
There's also an endorsement worth its correct weight. When Michael Mauboussin asked Daniel Kahneman what single thing an investor could do to improve performance, Kahneman's answer was a cheap notebook: write down what you expect to happen and why, so hindsight bias can't later convince you that you knew things you didn't. That's authority, not data. But it's authority from the person who mapped most of the biases involved.
Where that leaves it
The failure mode is real and well-documented. The measurement it requires is cheap: ten minutes per consequential decision. Each mechanism the practice relies on has independent support, and the one number circulating in its favor is the one thing I'd throw out. On that evidence I keep a journal, with a free template if you'd like to start. Run it for twenty decisions and score yourself; your own calibration curve will be more convincing than any study I could have cited.
Full disclosure: I built Reckon, an iOS app that runs this loop and does the scoring for you — prediction, confidence, check-ins, resolution, the delayed second look — so the measurement stays cheap enough that you actually keep doing it. One-time, no subscription. The template above works on paper; the app is for wanting it automatic.