How to keep a decision journal
A few years ago I was certain I'd predicted a project's failure. I remembered the conversation, the caveats I'd raised, the specific thing I said would go wrong. Then I found the actual email I'd written at the time. My "prediction" was two hedged sentences that could have been read as support. I hadn't predicted the failure; I'd absorbed the outcome and rewritten my memory around it.
That's not a personal quirk. Psychologists have watched people do this in controlled settings since the seventies. Fischhoff and Beyth had subjects forecast the outcomes of Nixon's 1972 trips to China and Moscow, then asked them months later what they'd predicted. The recalled predictions had drifted toward whatever actually happened, and the subjects were confident in the drifted versions. Your memory of your own judgment keeps editing itself to make you look reasonable.
Which is the whole case for a decision journal: it takes that unreliable narrator out of the loop. You write down what you expect before the outcome exists, and later the page tells you how good the call was — not your memory, which would have flattered you.
Here's how I keep mine. There's a free template if you want the fields ready to copy.
Write before the outcome
The entry you make at decision time needs four things:
The prediction, stated so it can fail. "This will probably work out" cannot be scored. "The contract runs its full term and I still ship the next version by September" can. If a stranger couldn't later mark your prediction right or wrong, it isn't a prediction yet.
A confidence number. Not "fairly sure", a percentage. This feels false at first; the precision seems unearned. But overconfidence survives on vague wording: when a 1,700-person survey asked people to put a number on common phrases, "a real possibility" meant anywhere from 20% to 80%, depending on who read it. The number isn't there because you know it's exactly 70%; it's there so that months of entries can be scored against reality. Twenty vague phrases teach you nothing, while twenty percentages become a calibration curve.
The reasoning, briefly. Two to five sentences on what you know, what you're assuming, and what you weighed. This is the part your future self can't reconstruct; after the outcome, you can only rebuild reasons that point toward what happened. I also note my state of mind. A decision made tired and rushed at 11pm is a different data point from the same decision made on a clear morning, and the pattern across entries is worth having.
A review date. Decisions don't announce their resolutions; they dissolve into daily life and you never formally learn whether you were right. Naming the date you'll come back — and putting it in your calendar — is what turns a note into a loop.
Check in while it's open
Most decision-journal advice treats a decision as two moments: the call and the verdict. But the information arrives in the middle. When something new lands, I add a dated line: what's new, whether it points for or against, and my confidence now.
These check-ins do two things. They timestamp the moment your thinking changed, so you can later see what sort of information moves you. And they stop a quiet failure mode: updating your beliefs gradually and then remembering that you "knew it all along." You didn't; the check-ins prove exactly when you knew it.
Score it, then look again
At resolution, three fields: what happened, whether the prediction was correct (yes, partly, no) and how satisfied you are, separately. Correctness and satisfaction diverge more often than you'd expect; you can call it right and hate the result, or blow the call and get lucky. Keeping them as two questions is what lets you eventually see whether your confidence tracks good outcomes or merely correct predictions. I also write one line on what I missed that was knowable at the time. Not what was unknowable — that's just the world being the world — but what was sitting in plain sight.
Then, thirty to sixty days later, the question the whole practice exists for: knowing only what I knew then, would I make the same call again? The delay matters. At resolution you're still inside the outcome's emotional weather; a month or two later you can judge the process instead of the result. When this answer disagrees with how the decision felt at resolution — and it regularly does — that disagreement is the most honest information in the journal.
What to log, and how much
Not everything. A journal that demands an entry for lunch dies in a week. My bar: decisions with consequences that unfold over weeks or longer, where I'd genuinely like to know if my judgment is any good — career moves, money above a threshold, health choices, meaningful product bets. One or two entries a week, ten minutes each.
One non-obvious tip: log some medium decisions, not only the giant ones. Big decisions resolve slowly and arrive rarely, so the feedback loop takes years. Medium ones resolve in weeks and teach you the same lessons about your own overconfidence, faster.
After fifty or so entries, patterns start to surface that introspection alone would never find: the confidence range where you're reliably overconfident, the domain where you're sharper than you believed, the specific flavor of new information that actually changes your mind. That's the payoff — the evidence for why this works is its own essay.
Full disclosure: I built Reckon, an iOS decision journal that runs exactly this loop — prediction, confidence, check-ins, resolution, the delayed second look — and does the calibration math for you, which is the part that makes the habit stick. One-time, no subscription. The paper template works; the app is for wanting the scoring automatic.