Brier score explained: how forecast accuracy is measured (with examples)
The Brier score is the average squared gap between a forecast and what happened. The formula, worked examples, what counts as a good score, the skill score, and how to score Polymarket and Kalshi prices.
In this article
- The Brier score formula
- A worked example
- What is a good Brier score?
- The Brier skill score
- Calibration and resolution: what the score rewards
- Brier score vs log score vs accuracy
- Original Brier score vs the binary version
- How to score prediction markets with the Brier score
- Score your own forecasts
- Brier score: frequently asked questions
- Keep reading
The Brier score measures how good probability forecasts are. For each forecast you take the probability you gave, subtract what actually happened (1 if it happened, 0 if it did not), square the gap and average over all your forecasts. Lower is better: 0 is perfect, and always saying 50% scores 0.25.
It was introduced by the meteorologist Glenn W. Brier in 1950 to score weather forecasts, and it is now the standard way to grade forecasters, forecasting tournaments and prediction markets.
The Brier score formula
For N yes-or-no questions:
Brier score = (1 / N) × Σ (f − o)²
- f is the forecast probability, from 0 to 1 (a 70% forecast is 0.7).
- o is the outcome: 1 if the event happened, 0 if it did not.
Each question contributes between 0 (you said 100% and it happened, or 0% and it did not) and 1 (you were certain and wrong). The average is the score.
A worked example
| Question | Forecast (f) | Happened? (o) | (f − o)² |
|---|---|---|---|
| Fed holds rates | 0.90 | Yes (1) | 0.01 |
| Team wins the final | 0.70 | Yes (1) | 0.09 |
| Bill passes this year | 0.60 | No (0) | 0.36 |
| Recession declared | 0.20 | No (0) | 0.04 |
| Candidate wins primary | 0.50 | Yes (1) | 0.25 |
Sum: 0.75. Divided by five questions: Brier score = 0.15.
The third row did the damage: a 60% forecast that did not happen costs 0.36, more than any other row. Squaring the error punishes confident misses much more than small ones.
What is a good Brier score?
| Brier score | What it means |
|---|---|
| 0 | Perfect: certain and right every time |
| about 0.05–0.10 | Very good on hard real-world questions |
| 0.25 | No skill: the score of always forecasting 50% |
| above 0.25 | Worse than a coin flip: overconfident and often wrong |
| 1 | Certain and wrong every time |
What counts as good depends on the questions. Forecasting whether the sun rises tomorrow earns a near-zero score with no skill at all, so compare scores only on the same set of questions, or use a skill score.
The Brier skill score
To see whether a forecaster beats a simple benchmark, compare their score with a reference forecast, such as always predicting the base rate:
Brier skill score = 1 − (your Brier score ÷ reference Brier score)
With the example above (0.15) and a 50% reference (0.25): 1 − 0.15 ÷ 0.25 = 0.40, a 40% improvement on guessing. A skill score of 0 means no better than the reference; negative means worse.
Calibration and resolution: what the score rewards
The Brier score can be split into parts, a decomposition published by Allan Murphy in 1973:
- Calibration (reliability): do your 70% forecasts come true about 70% of the time? Good calibration lowers the score.
- Resolution: do your forecasts move away from the base rate when they should? A forecaster who always says the average is calibrated but useless; resolution rewards confident, correct calls.
- Uncertainty: how hard the questions are. You cannot control this part.
So a low Brier score needs both honesty about your confidence and the nerve to forecast far from 50% when the evidence supports it.
Brier score vs log score vs accuracy
- Accuracy ("the favourite won 9 times out of 10") ignores how confident the forecast was. A 51% and a 99% forecast count the same.
- Brier score penalises errors by their square, so a confident miss costs at most 1.
- Log score uses the logarithm of the probability given to what happened. It punishes confident misses without limit: forecasting 0% for something that happens is infinitely bad. Metaculus builds its scores on log scoring (how Metaculus scores forecasters).
Both Brier and log scores are proper: you get the best expected score by reporting what you really believe, so there is no way to game them by shading your forecasts.
Original Brier score vs the binary version
Brier's 1950 formula sums the squared error over every possible outcome, so for a yes-or-no question each error is counted twice and the range is 0 to 2. Most people today use the binary version above, which runs from 0 to 1. If a source reports Brier scores above 0.5, check which version it uses.
How to score prediction markets with the Brier score
Treat the market price as the forecast. A contract at 72¢ is a 0.72 forecast. Then:
- Pick a fixed time before each market resolves: a month, a week or a day.
- Record the price of "Yes" at that time.
- Score it against the result, and average across many markets.
Analyses of resolved Polymarket markets done this way have reported Brier scores of roughly 0.03 to 0.09, getting better as resolution approaches, which is strong performance across thousands of questions. What the numbers say about markets in general: how accurate are prediction markets.
A few traps when you do this yourself: use the midpoint between bid and ask rather than the last trade on thin markets, normalise multi-outcome markets whose prices add up to more than 100% (why prices add up to more than 100%), and do not compare a score taken one day out with one taken a month out.
Score your own forecasts
The quickest way to get better at forecasting is to write down a probability before you trade and score yourself afterwards. A spreadsheet is enough: one column for the forecast, one for the result (1 or 0), and a third for (forecast − result)². The average of the third column is your Brier score. After 50 questions you will know whether your 70% calls really come true 70% of the time.
Brier score: frequently asked questions
Is a higher or lower Brier score better?
Lower. 0 is perfect; 0.25 is what you get by always saying 50%.
What is a good Brier score?
On hard real-world questions, anything clearly below 0.25 shows skill, and 0.1 or below is very good. Compare only on the same questions.
How is the Brier score different from accuracy?
Accuracy only checks whether the favourite won. The Brier score also rewards being confident when right and punishes being confident when wrong.
Who invented the Brier score?
Glenn W. Brier, a US Weather Bureau scientist, in a 1950 paper on verifying probability forecasts.
