Explain exactly how you turned a decade of scoring into a simulation engine. Show every seam. You may not hide anything embarrassing.
All the information is in the task.
1 · The data two sources, cross-checked row by row
Every score comes from two CC BY-SA sources: the taskmaster-uk-scores dataset and the Taskmaster Wiki, cross-checked against each other row by row. Where both had data they agreed on 4,460 of 4,471 scores; the wiki (validated against official episode totals) settled the 11 disagreements, resolved 225 ambiguous blanks, restored 48 tasks missing from the dataset, and contributed every disqualification, bonus point, and tiebreak on record. Series 21 was built from the wiki directly. The corpus: 105 contestants, 21 series, four Champion of Champions events, 1,047 tasks.
Go deeper: the reconciliation ledger
Both sources were normalised to one row per (contestant, task): 4,471 directly comparable scores, of which 4,460 agreed exactly. Resolution counts, all applied programmatically and logged:
11 score corrections (wiki value, validated against official episode totals) · 225 blanks resolved (dataset empty, wiki explicit – mostly DQs recorded as blanks) · 48 whole tasks recovered (present on the wiki, absent from the dataset; task IDs minted as W-S{series}-{ep}-{label}) · 221 disqualifications · 49 bonus points · 64 tiebreaks (excluded from the skill fit)
Every contestant page was self-validated: the sum of parsed task scores must equal the wiki's own stated series total, per contestant, before any row enters the corpus. All 105 reconcile exactly. The working files – scores_clean.csv, task_types.csv, wiki_enrichment_master.csv, fit_model.py – ship in the site repository's /fit folder.
2 · The model latent strength with honest shrinkage
Taskmaster points are relative: you get 5 for beating the four people in the room, whatever the room. So the model never compares raw points across series. Each task's scores become ranks; ranks become normal scores (the z-value of your percentile); and a contestant's skill is their average normal score, separately per task domain (physical, creative, mental, live, prize).
With only ~10 tasks per contestant per domain, raw averages would be noise. Every estimate is therefore shrunk toward the population mean in proportion to how little data supports it – empirical-Bayes partial pooling. A contestant is a distribution, not a number: alongside skill we fit a personal volatility (how erratic their task outcomes are) and a disqualification hazard (from 221 real DQs).
Cross-series comparability comes from the Champion of Champions events, which pin the twenty series' scales together with real head-to-head data. This is a Thurstonian latent-strength model, a sibling of the Plackett-Luce family used in ranking research; the current fit is by moment matching, with a full Bayesian upgrade on the roadmap.
Go deeper: the estimator, with formulas
Within a task with n scorers, contestant i's score becomes an average-tie rank r, then a percentile, then a normal score:
z = Φ−1(1 − r − 0.5n)
Overall skill is the shrunken mean of a contestant's normal scores, pulled toward the grand mean μ in proportion to sample size:
θi = Σz + κμni + κ κ = 4
Domain offsets shrink toward zero with κ_d = 6; personal volatility σᵢ is the residual standard deviation shrunk toward 1 with κ_σ = 8; the DQ hazard is beta-shrunk toward the pooled rate with prior weight 20. Team tasks are excluded from the individual fit (attribution is confounded); tasks recovered from the wiki without a type label enter the overall estimate with a zero domain offset. A simulated performance is then a single draw:
perf = θi + δi,d + σiε ε ~ N(0, 1)
This is moment-matching estimation of a Thurstone Case V-style latent-strength model. The roadmap upgrade is a full Bayesian Plackett–Luce fit with tie handling (Davidson extension), which changes the estimator but not the exported parameter contract.
3 · The simulation ten episodes of real tasks, 10,000 times
A simulated season is ten episodes of real tasks drawn from the historical pool: one prize task, three filmed tasks, one live task each. Every contestant's performance on a task is a draw: skill + domain offset + personal noise, with a DQ roll on top. Rank the draws, award 5-4-3-2-1 with ties sharing, exactly as the show does. The headline numbers come from 10,000 such seasons; the featured season you watch is one of them, chosen by the seed.
Go deeper: the season machinery
Each episode draws one prize task, three filmed tasks, and one live task from the historical pool without replacement within a season. Filmed tasks sample domains at the pool's observed mix:
physical 0.62 · creative 0.30 · mental 0.08
Scoring is competition ranking – 5-4-3-2-1 with tied performances sharing the higher value – and a disqualification forces score 0 and bottom position, rolled per contestant per task at their fitted hazard. Randomness is a seeded mulberry32 PRNG, which is what makes every season reproducible from its URL: the same seed replays the identical season, task for task. Headline probabilities aggregate 10,000 independent seasons with fresh task draws each time, so they include task-selection luck as well as performance luck.
4 · Does it work? the backtest, receipts included
Re-simulating all 21 real series: the model's favourite won …, the real champion received a mean title probability of … against a chance baseline of 20%, and predicted points correlate with actual finishing order at ρ ≈ 0.89 within series. Calibration by bucket:
The model is currently underconfident in the middle: contestants it gives 40-60% chances win more often than that. It under-claims rather than over-claims, which is the failure mode we prefer.
Go deeper: the backtest protocol
Each real series is re-simulated 3,000 times using its actual lineup and its actual solo, non-tiebreak task set – the comparison totals are recomputed over the same task subset, so simulated and actual points are apples to apples. The calibration table bins every contestant-series title probability (105 predictions, 21 winners) into 20-point buckets and compares mean prediction to observed win rate. The known weakness is honest: the test is in-sample. Leave-one-series-out is not a meaningful alternative here, because removing a series removes nearly all data about its five contestants – their predictions would collapse to the prior. True out-of-sample validation requires a season the fit has never seen, which is why Series 22 matters.
5 · What the model cannot know limitations, stated plainly
Stated plainly: it cannot simulate what anyone would bring to a prize task (it simulates how their prize-task instincts tend to be received). It assumes skill is fixed over a career. It knows nothing about chemistry, editing, or Greg's mood. The backtest is in-sample – the model saw the seasons it re-predicts – so it demonstrates consistency, not prophecy; Series 22 will be the first true out-of-sample test. Series 21 estimates rest on a single season of data and wear wide uncertainty accordingly. Task-type labels are partly machine-derived. The archetype lineups were found by k-means clustering on domain percentiles, volatility, and DQ rate; the algorithm found the families, and only the names are human. And one roster nuance: Champion of Champions fields are participants, not champions-by-right – Kiell Smith-Bynoe stood in for Mae Martin, so Mae's elite rating rests on Series 15 alone.
Task discrimination does this task know who's good?
Every solo task ranks five people. Discrimination asks: how closely did that ranking match those five contestants' overall fitted strengths? It's the correlation between within-task placement and career skill. Team tasks and tiebreaks are excluded – their outcomes aren't individual.
Near +1, the task is a pure test: the strong finish high, the weak finish low, almost every time. Near 0, outcomes are noise – a great leveller. Negative is the strange tier: tasks where being good at Taskmaster actively hurts, usually because the winning move is embarrassing, reckless, or requires ignoring an instinct strong players trust.
One honest caveat: each number rests on five data points, so individual scores are noisy – a +0.9 and a +0.7 are not meaningfully different. The extremes and the averages carry the signal, which is why series pages report a season's mean task diet rather than celebrating any single task's number.