The maths, written out.

Calibration and the Brier decomposition

For each settled forecast you stated a probability p, and the outcome is 1 if it happened and 0 if it did not. The Brier score is the mean squared error of those probabilities:

Brier = (1/N) · Σ (p_i − y_i)²

Lower is better, and the Murphy decomposition splits it into three parts that mean different things:

Brier = reliability − resolution + uncertainty

reliability  = Σ_b (n_b / N) · (f_b − p̄_b)²     lower is better
resolution   = Σ_b (n_b / N) · (f_b − ō)²        higher is better
uncertainty  = ō · (1 − ō)                       a property of the sample, not of you

Reliability is how far your stated probabilities sat from what actually happened — it is the diagonal in the reliability diagram. Resolution is how much your forecasts separated the outcomes away from the base rate. Uncertainty is the base rate's own variance; you cannot change it and it is not a compliment. The Brier skill score compares your Brier against always quoting the base rate, 1 − Brier / uncertainty.

The Wilson score interval

Each bin on the diagram holds n forecasts of which k came in, giving a raw frequency p̂ = k/n. A raw frequency swings wildly at small n, so the diagram draws a 95% Wilson score interval, which behaves sensibly near 0 and 1 where the textbook normal interval does not:

denom  = 1 + z²/n
centre = (p̂ + z²/(2n)) / denom
half   = z · √( p̂(1−p̂)/n + z²/(4n²) ) / denom
interval = [centre − half, centre + half],  z = 1.96 for 95%

A bin is scored only if it holds at least 8 forecasts. Thin bins are drawn hollow and amber and are labelled unscored — never merged into a neighbour, because merging would make the line look tidier than the data.

Closing line value, and removing the margin

The raw figure is the one almost everyone quotes, and it is biased:

Raw CLV = (odds_taken − closing_odds) / closing_odds

Every price in a market carries the bookmaker's margin. When a book widens before an event, all of the prices drift out together, so raw CLV turns positive for every outcome in that market — including the ones that lost. The honest version removes the margin across the whole market first. Write each closing price as an implied probability, q_i = 1/odds_i, and the overround is their sum minus one.

Proportional de-margining

p_i = q_i / Σ_j q_j

Proportional removes the margin in proportion to each price. It is simple and it systematically understates favourites and overstates longshots.

Shin de-margining

p_i(z) = ( √( z² + 4(1−z)·q_i² ) − z ) / ( 2(1−z) )

Solve for z ∈ [0, 0.5] such that Σ_i p_i(z) = 1

Shin's model corrects for the favourite–longshot bias, so it pushes more of the margin off longshots and less off favourites. The toggle between the two is offered wherever a full market exists.

True CLV is then measured against the fair closing probability of the selection:

True CLV = odds_taken · fair_closing_probability − 1

Where the closing odds for every outcome of the market were not supplied, the margin cannot be removed. Those rows are marked margin not removable, true CLV is reported as unavailable with a count, and nothing is ever imputed.

The bootstrap

To ask whether a result is distinguishable from luck, Hindcast resamples the record itself — no distributional assumption, no appeal to a formula that assumes a tidy world:

Repeat 10,000 times:
   draw N rows from the record, with replacement
   recompute the statistic on the drawn sample
     · ROI      = Σ profit / Σ stake
     · mean CLV = mean of true CLV over rows with a full market

Take the 2.5th and 97.5th percentiles → a 95% bootstrap interval.

The resampling runs in a background worker, so the interface stays responsive while ten thousand samples are drawn. The point estimate is the statistic on your actual record; the interval is where the resamples fell.

The sample-size figure inverts the interval's own width. If the half-width scales as 1/√n, then n* = n · (half-width / point estimate)² is the size at which the interval would just stop containing zero, holding your observed edge fixed. It is an arithmetic statement about this sample, not a forecast that the edge will persist.

What this cannot tell you

The short version, with the full page linked below.

Survivorship in self-reported records

A record you compiled yourself contains the forecasts you chose to write down. If you logged the ones you felt good about and quietly forgot the rest, no amount of arithmetic here can recover the missing rows — the maths is only ever as honest as the ledger it is handed. Publish in advance and each row is stamped before the event; that is the only structural defence, and it is why the Record tier exists.

Beating the closing line is not beating the market

Closing line value measures whether you took a better price than the market settled at. It is a stronger signal than profit, and it is still not the same as a bankable edge: you can consistently beat the close and still be restricted, limited, or closed, and you can beat the close on a market where the price you actually got was available to everyone for a minute. A positive true CLV is evidence about your pricing, not a promise about your P&L.

A good Brier score on a soft market is not evidence of anything

Calibration is scored against the outcomes, never against a market. A well-run forecast on a market that nobody prices sharply can be beautifully calibrated while proving nothing about skill — there was no competition to beat. Calibration answers "did you mean what you said"; it does not answer "were you right in a way that could have been wrong".