# ADR-042 — odme-dtalite: DTALite's static ODME as one more guarded T2 estimator row

**Status:** accepted (shipped in v0.2)
**File:** `docs/design/adr-042-odme-dtalite.md`

## Context — the T2 thesis, and a second engine-in-the-loop estimator

T2 (ADR-002) ships a demand-free estimation contract: anything that emits an OD
matrix is scored by the identical **pinned-bfw certifier** (`metrics/estimation.py`),
which recomputes count-fit and OD-fit from the emitted OD through *its own* reference
assignment (P1, ADR-002 Decision 2). `spsa-sumo` (ADR-028) was the first *guarded*
estimator riding a production engine, and — crucially — it did so with **zero task,
runner, or certifier changes** (the adr-028 ideal): the estimator emits, the unchanged
certifier scores.

DTALite 0.8.1 (Zhou & Taylor 2014, canon `zhou2014dtalite` — the tool paper whose title
names "fast model evaluation and *calibration*") hides an Origin-Destination Matrix
Estimation routine (`performODME`) **inside the same static `assignment()` entry** the
`dtalite-tap` T1 adapter (ADR-029) already wraps: it runs in `TAPLite.cpp`'s
`AssignmentAPI()` right after the Frank-Wolfe loop when `settings.csv` carries
`odme_mode=1`. There is NO separate ODME API. It is a gradient-descent OD calibrator —
given a seed OD, a target OD, and sensor link counts (`obs_volume`), it re-weights the
route-flows the base assignment discovered so the modeled link volumes approach the
counts. This ADR ships it as `odme-dtalite`, the second engine-in-the-loop T2 estimator,
under the same adr-028 zero-certifier-change ideal.

## Decision 1 — a GUARDED STATIC estimator on the UNCHANGED certifier

`odme-dtalite` is a `DtaliteODMEEstimator(ODEstimator)` registered in
`ESTIMATOR_REGISTRY` behind a guarded import (Decision 6). It emits an OD matrix
through `ODResultBundle.final.od_matrix` — the SAME channel `spsa`/`gls`/`spsa-sumo`
use — and the EXISTING pinned-bfw `ODCertifier` scores it unchanged. **No
`estimation/base.py`, `metrics/estimation.py` (`ODCertifier`), `experiments/runner.py`,
or `EstimationTask` code is touched.** The scope-probe confirmed this is possible:
ODME lives on the *static* TAPLite path (never `SimulationAPI`/`trajectory.csv`), so the
OD it emits is certifiable through the exact same declared-BPR pinned-bfw pin every
static T2 row uses — a **guarded static estimator**, not a D2 observational certificate
(which would only have been forced if ODME needed a re-simulated held-out count).

A **DTALite-pinned certifier was considered and REJECTED**, for the same reason
adr-028 rejected a sumo-pinned one: scoring estimators against the very engine one of
them runs in the loop is a conflict of interest, and the whole T2 point is that the
certifier is model-blind. Keeping the pinned-bfw certifier makes the standard
self-vs-certified honesty diff MEASURE the engine-in-the-loop bias (Decision 5).

## Decision 2 — `route_output=1` is REQUIRED (the non-obvious, measured pin)

This is the load-bearing finding of the sprint, and it is **not** in the design dossier
— it was caught by the mandatory measure-first pilot (below). ODME does not read the
Frank-Wolfe `MainVolume` when it computes each observed link's *modeled* volume; it
reconstructs that volume from the stored **route/path history**. With DTALite's lean
`route_output=0` (the `dtalite-tap` T1 default), that history is never populated, so the
reconstruction collapses to ~0 on every sensor. ODME then sees a fake, enormous deficit
on every observed link and inflates every OD cell to its box ceiling.

Measured on the Sioux Falls marquee anchor (below), with `route_output=0`:
the emitted OD total demand is **+40 %** over truth, `od_rmse` ~2× WORSE than the prior,
and the certified `obs_count_rmse` ~7× WORSE — a *degenerate* estimator that would have
silently shipped had the pilot not measured the certified before/after. Flipping to
`route_output=1` restores a faithful reconstruction (the engine's internal predicted
volume matches the observed count to ~1.0 ratio) and turns the row non-degenerate
(Decision 3). `settings.csv` therefore pins `route_output=1` AND `odme_mode=1`; a
structural test fails if either reverts. `route_output=1` is part of the pinned
estimator identity.

## Decision 3 — the marquee anchor is Sioux Falls scale (the tol=1 floor), measured

The pilot (ruling: *pilot-and-decide the anchor BEFORE the full build*) proved a
non-degenerate recovery exists. **Marquee anchor:** `siouxfalls` UE (24 zones / 76 links
/ 528 OD pairs, BPR power-4), clean counts, sensors random coverage 0.5 (38 links),
held-out coverage 0.2, stale prior `cv=0.3`, seed 7. Through the UNCHANGED pinned-bfw
certifier:

| metric | prior baseline | odme-dtalite | ratio |
|---|---|---|---|
| `obs_count_rmse` | 994.8 | 365.3 | **0.37×** |
| `heldout_count_rmse` (ranking) | 816.5 | 556.6 | **0.68×** |
| `od_rmse` | 267.4 | 264.5 | 0.99× |
| `total_demand_error` | 0.014 | −0.002 | — |

`od_feasible = 1`, `certificate_converged = 1`, `certificate_gap < 1e-6`, and the ODME
gradient descent ran **69 iterations** (never a "Convergence reached after 0 iterations"
no-op). This is a real "estimator beats prior" row: ODME measurably improves BOTH the
observed count fit it calibrates AND the ranking held-out count fit. `od_rmse` barely
moves — ODME is a count-matcher and Sioux Falls OD is not linearly identifiable from
these sensors (`od_identifiable = 0`, reported, honest).

**Why Sioux Falls, not Braess.** DTALite's ODME convergence tolerance is a HARDCODED
*absolute* gradient-norm floor `tol=1` (Decision 4). On Braess-scale demand (~6), a
realistic count mismatch never clears the floor and ODME no-ops with "0 iterations"
(measured). The marquee anchor MUST be at a demand magnitude where realistic deviations
clear the fixed floor — Sioux Falls (OD mean ~683, min 100) does; Braess does not. This
is the same A4 scale choice `dtalite-tap` made.

## Decision 4 — the disclosed engine envelope (part of the pinned 0.8.1 identity)

Three hardcoded, non-configurable engine behaviors bound what this row can do. They are
disclosed as HONEST scope boundaries — reproducibility-good, like marouter's hardcoded
linear vdf (ADR-027) — not defects:

1. **A hardcoded `[0.5·min(seed,target), 1.5·max(seed,target)]` box** baked into the
   line search, no settings-file exposure. With `demand_target = prior`, the estimator
   can only recover a truth cell within ~2× of the prior. Measured: with a large count
   mismatch the emitted cell lands at EXACTLY `1.5×` and sits there for the rest of the
   descent (a hard wall, not a soft penalty). `demand_target_frac` (default 1.0) is a
   documented **one-sided** dial: `demand_target.csv = prior · frac`, so `>1` raises the
   upper bound, `<1` lowers the lower bound (never both), at the cost of biasing the
   (also hardcoded) OD-regularization pull toward the inflated target. It is a hashed
   estimator-identity factor.
2. **Hardcoded penalty weights** (`w_link=0.1`, `w_od=0.01`, `w_vmt=1e-6`) and the
   ABSOLUTE `tol=1` convergence floor (Decision 3). Only `odme_mode`/`odme_vmt` are
   settings-configurable; the rest are C++ literals, along with the fixed 400-iteration
   descent. `budget.iterations` therefore maps to the base FW `number_of_iterations`
   only — the ODME descent itself is not budget-controllable, which is disclosed, not
   hidden.
3. **A route-history fidelity caveat.** ODME re-weights routes the prior BFW/AON
   assignment already loaded; it cannot invent flow on an unloaded route. `route_output=1`
   (Decision 2) is what makes the captured routes' reconstruction faithful; a genuinely
   unloaded observed link stays a blind spot.

**Scope:** single-mode "auto" demand-only, matching `dtalite-tap`'s sprint-1 scope.
Native multiclass ODME (`mode_type.csv`) is a named deferred follow-up. The scenario
envelope IS the T1 adapter's — refusals (nonzero fixed cost / toll, sub-0.1 capacity
clamp, the SUE-family task fields) are delegated to
`DTALiteTapModel._refuse_unrepresentable` verbatim (the spsa-sumo precedent). Note
power≠1 is representable here (DTALite's VDF is the repo BPR exactly), unlike the
marouter-backed `spsa-sumo` which refused power-4.

## Decision 5 — `link_performance.csv` is corrupt under ODME; source the OD from `od_performance.csv` ONLY

Measured: under `odme_mode=1` a lossy post-ODME "final synchronization" reconstructs the
link volumes from the route-history representation rather than the Frank-Wolfe
`MainVolume`, so they no longer conserve OD demand. The illustrative signature — a link
reporting `volume=0` while its own `travel_time` implies a nonzero flow, and total
reconstructed link flow not matching total OD demand even at 0 gradient steps — was
measured on a **partially-loaded** two-route toy net (the design-probe `designB`
instance), and reproduces generally under the reconstruction-starved `route_output=0`
regime. It does NOT reproduce uniformly on the SHIPPED `route_output=1` + fully-loaded
marquee (on Sioux Falls 0/76 links show the `volume=0`/`travel_time>0` inversion — every
link is loaded). The
substantive invariant is config-independent and is what the row relies on:
`link_performance.csv` is **structurally never read** (no `open()` touches it; the emitted
OD is byte-identical after deleting the file, pinned by a test), so whatever corruption
it carries is unreachable by the certifier regardless of scenario or settings. The OD
estimate is therefore sourced **exclusively** from `od_performance.csv`'s `volume` column
(`= MDODflow`), which is unaffected. And the `dtalite-tap`
`odme_mode=0` read-back gates (the A2 cost-match / per-origin mass-gate / echo-check) are
NEVER reused on an ODME run — they are validated only for `odme_mode=0`. The read-back
transports `dtalite-tap`'s "every repo link matched exactly once" discipline to the
OD-pair domain: every prior-support pair present (completeness), a pair's route rows
consistent, no off-support/phantom pair, every volume finite and non-negative — else
`RuntimeError` (never trust `returncode` alone; the engine exits 0 on garbage).

## Decision 6 — the anti-laundering property, and the honesty diff as a measured bias

The **anti-laundering property is sacred and holds by construction.**
`ODCertifier.certify()` (read in full) accepts only a raw `(Z,Z)` numpy OD and re-runs
its OWN pinned bfw solve — there is no DTALite import, no engine branch anywhere in that
file. The ONLY channel from `odme-dtalite` to the harness is
`ODResultBundle.final.od_matrix`. So even though DTALite's own `link_performance.csv` is
corrupt (Decision 5), that corruption is *structurally unreachable* by the certifier. A
buggy or adversarial DTALite run can at worst emit a *bad* OD that certifies honestly as
a no-improvement row — it CANNOT forge a good certificate.

This is pinned from both sides by tests (Decision 7). Measured on the marquee: DTALite's
OWN self-reported count fit (from its stalled-assignment, box-clamped predicted volumes)
is `obs_count_rmse ≈ 8`, while the pinned-bfw certifier scores the emitted OD at
`≈ 365`. The certifier ignored the rosy self-report entirely. That large self-vs-certified
diff MEASURES the engine-in-the-loop bias (the `spsa-sumo` reframing: DTALite descends
its own stalled law under a hardcoded box; the certificate re-assigns under the declared
BPR), reported as provenance, never a bound or a score.

## Decision 7 — the FIRST guarded ODME estimator (guard shape, subprocess discipline)

Guarded byte-parallel to the `spsa-sumo` block (and the T1 model guard): a module-top
`find_spec("DTALite")` probe raises `ModuleNotFoundError(name="DTALite")`, swallowed by
exact name in `estimation/__init__.py` — the exact CAPITAL `DTALite` name (the wheel's
module name, ADR-029), never imported in-host (the wheel prints a banner and ctypes-loads
an OpenMP engine on import). Subprocess discipline is the adr-029 contract verbatim:
subprocess-only (`[sys.executable, "-c", "import DTALite; DTALite.assignment()"]`),
`stdin=DEVNULL`, `OMP_NUM_THREADS=1` (a correctness pin — the assignment/ODME path was
measured race-free, bit-identical across 21/21 runs at OMP∈{1,4}, but the pin is kept as
defense-in-depth), tempdir-per-run with `finally` cleanup, one wall deadline threaded
across write → subprocess → parse, `returncode` never trusted (success = read-back), and
an engine crash/timeout RAISES `RuntimeError` (never a false estimate). An `sp_calls`-only
budget is refused up front (the engine hides its Dijkstra count — the adr-027/028/029
pattern).

## Determinism / wall math (measured on this box, DTALite 0.8.1 wheel)

The assignment+ODME path is byte-deterministic under `OMP_NUM_THREADS=1`: the marquee's
`od_performance.csv` is md5-identical across 3 reruns, so the estimator is
`deterministic=True`, `seedable=False` (no engine seed to pin — the RngBundle root seed
lands in the bundle as provenance). The ODME subprocess wall at the marquee scale is
~0.4–2.0 s (base `number_of_iterations` 20→400); the row rides the EXISTING `dtalite` CI
job (same 0.8.1 pin), adding `tests/test_odme_dtalite.py` to that job's explicit file
list and `odme-dtalite` to its notebook `-k` filter. No new CI job.

## Honest sourcing

`zhou2014dtalite` anchors the DTALite **software lineage** (tool-paper discipline, the
lopez2018/adr-029 precedent): its title names calibration, of which ODME is an instance,
so no new bib entry is needed. The row validates the ADAPTER + engine fidelity, never the
paper's numerics. The engine version + the ODME settings (`odme_mode=1`, `route_output=1`,
the hardcoded weights/box/tol it implies) are part of the estimator's pinned identity.
This is the same wheel and `.so` (md5 `e179ed66…`, S4-verified) as `dtalite-tap` and
`dtalite-simulation` — three rows, three engine entries, one pinned wheel.

## Consequences

* One more T2 leaderboard row, on the SAME pinned-bfw certifier and the SAME separate-no
  table as every static estimator (directly comparable — unlike the EDOC observational
  rows). Additive; the golden Braess content hash `cf00f411…` is byte-identical.
* A new, disclosed pattern: a guarded estimator whose engine has a hardcoded
  representability box + convergence floor, honestly bounded rather than tuned. The
  marquee is chosen to clear the floor; a small-demand anchor no-ops (disclosed).
* The measure-first pilot earned its keep: the `route_output=0` degeneration would have
  shipped a silently-broken always-inflating estimator. `route_output=1` is now pinned
  and tested.

## Adversarial review — measured hazards, all disclosed, none blocking

* **`route_output=0` degeneration (Decision 2)** — the sprint's headline finding, caught
  by the pilot, pinned by a structural settings test.
* **link_performance corruption (Decision 5)** — never read; a test deletes it post-run
  and re-parses `od_performance.csv` to byte-identical OD.
* **the `[0.5,1.5]×` box + `tol=1` floor (Decisions 3,4)** — disclosed envelope; the
  marquee clears the floor with 69 iterations, and `demand_target_frac` is a documented
  one-sided dial.
* **anti-laundering (Decision 6)** — pinned from both sides: the certifier ignores
  DTALite's rosy self-report (measured 8 vs certified 365), and a poisoned OD certifies
  feasible-but-worse.