# TABenchmark tutorials

One notebook per benchmark unit — every registered model and estimator, plus each
parallel-track solver — showing how to run it through the **public** `tabench` API and,
above all, how the harness *certifies* the result.

**Follow them in order.** Notebooks are numbered **simple → complex** (PI directive,
2026-07-16): `tutorials/<NN>-<track>/<MM>-<unit>.ipynb`. The numeric prefixes make the
GitHub directory listing itself read as the learning path — start at `01-static/01-aon`
and walk up the vintage ladder. Track folders mirror the `src/tabench` parallel modules,
so a track's notebooks and its code land in the same review.

## The honesty rule

**A notebook never claims a number it does not compute in that cell.** Every scored
quantity is recomputed live by the track's P1 evaluator from the outputs the model
emitted, in the cell where it is claimed; model self-reports are shown only as
provenance and diffed against the certificate as an honesty check. Analytic anchors are
recomputed, never quoted. Plots draw only quantities certified in the notebook — through
the house visualizer `tabench.viz` where the artifact is road link flows / an OD matrix;
non-road tracks (cumulative diagrams, occupancy series, `(x,t)` fields, transit
multigraphs) plot the certified artifact directly with plain matplotlib, stating the
reason in the Visualize cell's markdown (adr-035 Decision C).

## Running them

```bash
pip install -e '.[dev,viz,tutorials]'                # nbclient/nbformat/ipykernel + matplotlib
jupyter lab tutorials/01-static/05-bfw.ipynb         # interactive
jupyter nbconvert --to notebook --execute tutorials/01-static/05-bfw.ipynb --stdout  # headless
```

Notebooks are committed **executed** — every non-empty code cell carries the outputs and
the strictly sequential execution count of one clean top-to-bottom run, so the committed
notebook is itself the rendered tutorial. Drift safety does NOT come from output
identity: CI re-executes every notebook from a cleared state, and the gate is *execution
success plus the in-notebook certified asserts*. The one exemption ships stripped: the
matsim notebook (JAVA-only toolchain, adr-039), whose executability the matsim CI job
proves instead. **Every CI leg enforces this.**
`tests/test_tutorials.py` runs unconditionally on existence + executed-state + metadata +
numbering, no env flag needed. The notebook-EXECUTION test (`test_notebook_executes`) is
wired in too: the core job's Python 3.12 leg installs `.[dev,viz,tutorials]` and sets
`TABENCH_RUN_TUTORIALS=1`, re-executing every unguarded notebook from a cleared kernel
(the 3.10 leg skips it — redundant kernel-start cost, no extra signal); each of the
torch / sumo / dtalite legs installs its own extra plus `tutorials` and runs
`tests/test_tutorials.py` with a `-k` filter scoped to its own extra-gated notebooks. A
notebook that raises, or whose in-cell certified `assert` fails, fails CI — run
`TABENCH_RUN_TUTORIALS=1 pytest tests/test_tutorials.py -k executes` locally for the same
guarantee.

## Enforcement

`tests/test_tutorials.py` makes "a new model ships a tutorial" mechanical: every key of
`MODEL_REGISTRY ∪ ESTIMATOR_REGISTRY ∪ DYNAMIC_ESTIMATOR_REGISTRY` and every unit in the
import-anchored 11-unit parallel-track manifest must map to a notebook (or a same-ADR
`covers` sibling), notebooks must be committed executed (matsim excepted), and `metadata.tabench` must match the
folder. A registered unit with no notebook turns the suite red. Guarded units register
only where their extra is installed, so enforcement is automatically environment-correct.

The coverage gate matches a unit `u` at `tutorials/*/[0-9][0-9]-{u}.ipynb`, and
`metadata.tabench` carries the **bare** names — `unit` = the registry key, `track` = the
folder name *sans* its numeric prefix — so a renumber never touches notebook content.
`test_notebook_numbering_is_ordered` binds the ordering: each track folder carries its
canonical number (unique across folders), and within a folder the file numbers are unique
and contiguous from `01`, so the learning path can never silently drift or collide.

Each notebook carries `metadata.tabench = {track, unit, requires_extra, covers}`; the CI
executor and the coverage gate key off that, not off filenames.

### Same-ADR `covers` folds

Mechanical folds — `metadata.tabench.covers` mirrors `tests/test_tutorials.py::_COVERS`
exactly (test-pinned, see Enforcement above):

| notebook | also certifies | why |
|---|---|---|
| `03-estimation/01-gls` | `prior` | ADR-002 static T2: the stale prior is the baseline the GLS estimator improves on |
| `03-estimation/07-od-dynamic` | `od-dynamic-sim`, `od-dynamic-seq`, `prior-profile` | ADR-023 within-day dynamic estimation — one notebook, three estimator variants |

`08-tdta/01-pm-td` is NOT a registry fold (no `pm-td-ue`/`pm-td-so` enforced units exist,
`metadata.tabench.covers == []`) — it is one notebook exercising both of `solve_td_ue`'s
and `solve_td_so`'s objectives over the single ADR-031 path-marginal task.

## Index (the learning path, in order)

`extra` blank = core install. Runtimes are wall-clock on the author's box (kernel start
~2–3 s dominates the small ones); treat them as order-of-magnitude, re-measured in CI.
Every enforced unit and every track tour below has shipped — the ToC is complete, and
`_ALLOWLIST` in `tests/test_tutorials.py` is empty (see Enforcement above).

### `01-static/` — road static assignment (T1), the model-evolution ladder
| # | unit | extra | runtime | status |
|---|---|---|---|---|
| 01 | `aon` | | 3.9 s | shipped |
| 02 | `msa` | | 4.0 s | shipped |
| 03 | `fw` | | 4.0 s | shipped |
| 04 | `cfw` | | 3.5 s | shipped |
| 05 | `bfw` | | 3.9 s | shipped |
| 06 | `gp` | | 3.9 s | shipped |
| 07 | `oba` | | 3.8 s | shipped |
| 08 | `algb` | | 3.8 s | shipped |
| 09 | `tapas` | | 3.6 s | shipped |
| 10 | `sue-msa` | | 4.2 s | shipped |
| 11 | `sue-probit-msa` | | 4.3 s | shipped |
| 12 | `so-bfw` | | 3.7 s | shipped |
| 13 | `fw-elastic` | | 3.9 s | shipped |
| 14 | `evans` | | 3.9 s | shipped |
| 15 | `br-ue` | | 3.6 s | shipped |
| 16 | `sc-tap` | | 3.5 s | shipped |
| 17 | `vi-asym` | | 3.9 s | shipped |
| 18 | `multiclass` | | 3.6 s | shipped |
| 19 | `learned-surrogate` | | 3.6 s | shipped |

### `02-day-to-day/` — day-to-day dynamics
| # | unit | extra | runtime | status |
|---|---|---|---|---|
| 01 | `dtd-swap` | | 6.6 s | shipped |
| 02 | `dtd-swap-sue` | | 5.5 s | shipped |
| 03 | `dtd-link` | | 5.1 s | shipped |
| 04 | `dtd-friesz` | | 5.2 s | shipped |
| 05 | `dtd-horowitz` | | 5.7 s | shipped |
| 06 | `dtd-stochastic` | | 7.1 s | shipped |
| 07 | `dtd-unifying` | | 5.9 s | shipped |
| 08 | `dtd-cumlog` | | 6.0 s | shipped |

### `03-estimation/` — T2 OD estimation
| # | unit | extra | runtime | status |
|---|---|---|---|---|
| 01 | `gls` (covers `prior`) | | 7.7 s | shipped |
| 02 | `spiess` | | 12.5 s | shipped |
| 03 | `vzw-entropy` | | 7.3 s | shipped |
| 04 | `od-congested` | | 10.3 s | shipped |
| 05 | `spsa` | | 12.3 s | shipped |
| 06 | `od-kalman` | | 7.1 s | shipped |
| 07 | `od-dynamic` (covers `od-dynamic-sim`, `od-dynamic-seq`, `prior-profile`) | | 5.3 s | shipped |

### `04-transit/`
| # | unit | extra | runtime | status |
|---|---|---|---|---|
| 01 | `transit-strategy` | | 5.2 s | shipped |

### `05-dnl/` — dynamic network loading
| # | unit | extra | runtime | status |
|---|---|---|---|---|
| 01 | `ctm` | | 5.5 s | shipped |
| 02 | `ltm` | | 4.8 s | shipped |
| 03 | `godunov` | | 5.3 s | shipped |
| 04 | `node-model` | | 5.0 s | shipped |

### `06-bottleneck/` · `07-dta/` · `08-tdta/` — analytic dynamics
| folder / # | units | extra | runtime | status |
|---|---|---|---|---|
| `06-bottleneck` 01 | `vickrey` | | 5.0 s | shipped |
| `06-bottleneck` 02 | `vi-due` | | 5.2 s | shipped |
| `07-dta` 01 | `merchant-nemhauser` | | 5.5 s | shipped |
| `07-dta` 02 | `lp-so-dta` | | 5.3 s | shipped |
| `08-tdta` 01 | `pm-td` | | 37.5 s | shipped |

### `09-newell/`
| # | unit | extra | runtime | status |
|---|---|---|---|---|
| 01 | `newell-3det` | | 5.4 s | shipped |

### `10-learned/` — optional-extra learned models
| # | unit | extra | runtime | status |
|---|---|---|---|---|
| 01 | `implicit-ue-nn` | torch | 19 s | shipped |
| 02 | `het-gnn` | torch | 71 s | shipped |

### `11-external/` — external-engine adapters
| # | unit | extra | runtime | status |
|---|---|---|---|---|
| 01 | `sumo-marouter` | sumo | 8 s | shipped |
| 02 | `sumo-duaiterate` | sumo | 12 s | shipped |
| 03 | `dtalite-tap` | dtalite | 7 s | shipped |
| 04 | `spsa-sumo` | sumo | 23 s | shipped |

### `12-data/` · `13-experiments/` — tours (outside the enforced unit set)
| folder / # | notebook | extra | runtime | status |
|---|---|---|---|---|
| `12-data` 01 | `xu2024` | network download | 14 s | shipped |
| `12-data` 02 | `bo4mob` | network download | 7 s | shipped |
| `13-experiments` 01 | `profiles` | | 5 s | shipped |

Data-track notebooks never default-prefetch: `xu2024`/`bo4mob` fetch (checksummed) or
raise a clear diagnostic if the network is unreachable, per each notebook's own guard.