`odme-dtalite` — DTALite’s static ODME as one more guarded T2 estimator (Zhou & Taylor 2014)¶

What. odme-dtalite (T2, ADR-042) is the estimation track’s second engine-in-the-loop estimator and its first ODME row. DTALite 0.8.1’s PyPI wheel hides an Origin-Destination Matrix Estimation routine (performODME) inside the same static assignment() entry the dtalite-tap T1 adapter (adr-029) already wraps — it runs when settings.csv carries odme_mode=1. Given a seed OD, a target OD, and sensor link counts (obs_volume), it re-weights the route-flows the base Frank-Wolfe assignment discovered so the modeled link volumes approach the counts ([zhou2014dtalite], docs/REFERENCES.md).

Why it is in the benchmark. T2’s whole thesis is that ANY OD-emitting method is scored by the SAME pinned-bfw certifier. odme-dtalite ships as a GUARDED STATIC estimator riding that UNCHANGED certifier (the adr-028 ideal: zero task, runner, or certifier changes) — the estimator emits an OD, the certifier re-assigns it under the declared BPR. It is a row, not a new question.

How this notebook is graded¶

A notebook never claims a number it does not compute in that cell. Every scored quantity below — od_feasible, certificate_gap, obs_count_rmse, heldout_count_rmse — comes from the SAME ODCertifier every other T2 estimator here is scored by, re-assigning the emitted OD through its own pinned bfw reference, never from odme-dtalite’s self-report (README, Certified, not self-reported).

# Setup. `odme-dtalite` needs the optional `dtalite` extra
# (`pip install tabench[dtalite]`). We probe it WITHOUT importing it: `import
# DTALite` prints a banner and ctypes-loads an OpenMP engine into the host
# (adr-029), so the estimator only ever runs it in a throwaway subprocess.
#
# The inline backend renders figures headlessly so CI can execute tutorials
# without a display. NEVER matplotlib.use("Agg") in-kernel -- it silently
# suppresses inline capture.
%matplotlib inline
import importlib.util

if importlib.util.find_spec("DTALite") is None:
    raise ModuleNotFoundError(
        "odme-dtalite needs the optional 'dtalite' extra: pip install tabench[dtalite]",
        name="DTALite",
    )

import matplotlib.pyplot as plt

from tabench import Budget, PriorBaseline, run_estimation_experiment
from tabench.data import load_scenario
from tabench.estimation import DtaliteODMEEstimator

# THE marquee anchor (adr-042): siouxfalls UE (BPR power-4, 528 OD pairs), clean
# counts, sensors random coverage 0.5, held-out coverage 0.2, stale prior cv=0.3.
# Sioux-Falls scale is DELIBERATE: DTALite's ODME has a hardcoded ABSOLUTE
# convergence floor (tol=1), so Braess-scale demand (~6) would no-op with "0
# iterations". At this scale realistic deviations clear the floor.
MARQUEE = {
    "sensors": {"kind": "random", "coverage": 0.5},
    "heldout": {"kind": "random", "coverage": 0.2},
    "noise": "none", "n_periods": 1, "prior": {"kind": "stale", "cv": 0.3},
}
SEED = 7
scenario = load_scenario("siouxfalls")
print(f"scenario: {scenario.name}, {scenario.network.n_zones} zones / "
      f"{scenario.network.n_links} links, BPR power {scenario.network.power[0]:.0f}")

scenario: siouxfalls, 24 zones / 76 links, BPR power 4

The recovery anchor, certified through the UNCHANGED pinned-`bfw` certifier¶

Zero changes to the task, runner, or certifier were needed for this row: ODCertifier re-assigns the emitted OD through its OWN pinned bfw reference regardless of which engine produced it — the whole point of T2. One non-obvious engine pin makes it work: odme-dtalite writes route_output=1 so DTALite populates the route history its ODME reconstruction reads (with the lean route_output=0, the reconstruction collapses and the OD inflates ~40 %, a degenerate estimate — measured, adr-042).

res = run_estimation_experiment(
    scenario, [PriorBaseline(), DtaliteODMEEstimator()], Budget(iterations=100),
    seed=SEED, macroreps=1, estimation=MARQUEE,
)
last = {row["estimator"]: row for row in res.rows}
prior, odme = last["prior"], last["odme-dtalite"]

print(f"certified od_feasible               : {odme['od_feasible']:.0f}")
print(f"certificate_converged               : {odme['certificate_converged']:.0f}")
print(f"certificate_gap                     : {float(odme['certificate_gap']):.2e}")
print(f"obs_count_rmse    odme / prior       : {float(odme['obs_count_rmse']):.1f} / "
      f"{float(prior['obs_count_rmse']):.1f}")
print(f"heldout_count_rmse odme / prior (rank): {float(odme['heldout_count_rmse']):.1f} / "
      f"{float(prior['heldout_count_rmse']):.1f}")
print(f"od_rmse           odme / prior       : {float(odme['od_rmse']):.1f} / "
      f"{float(prior['od_rmse']):.1f}")

# Certified, feasible, and a real improves-on-prior on the count fit it calibrates
# AND the ranking held-out count fit (loose bounds; the .so can shift the decimals).
assert odme["od_feasible"] == 1.0 and odme["certificate_converged"] == 1.0
assert float(odme["obs_count_rmse"]) < 0.6 * float(prior["obs_count_rmse"])
assert float(odme["heldout_count_rmse"]) < 0.9 * float(prior["heldout_count_rmse"])

certified od_feasible               : 1
certificate_converged               : 1
certificate_gap                     : 9.51e-07
obs_count_rmse    odme / prior       : 365.3 / 994.8
heldout_count_rmse odme / prior (rank): 556.6 / 816.5
od_rmse           odme / prior       : 264.5 / 267.4

The disclosed envelope, and that the ODME descent actually fired¶

DTALite’s ODME is bounded by hardcoded, non-configurable engine constants (part of the pinned 0.8.1 identity, like marouter’s linear vdf): a [0.5·min, 1.5·max]-of-(seed,target) recovery box, penalty weights w_link=0.1 / w_od=0.01, and the absolute tol=1 convergence floor plus a fixed 400-iteration descent. demand_target_frac (default 1.0) is the one documented dial — a one-sided widening of the box (>1 raises the upper bound only). od_rmse barely moves because Sioux Falls OD is not linearly identifiable from these sensors (od_identifiable = 0, reported) — odme-dtalite is a count-matcher, honestly.

bundle = res.bundles[("odme-dtalite", "m0")]
odme_iters = float(bundle.trace.final.self_report["engine_odme_iterations"])
print(f"ODME gradient iterations executed : {odme_iters:.0f}  (>0: cleared the tol=1 floor)")
print(f"od_identifiable                   : {odme['od_identifiable']:.0f}  "
      "(0 -> od_rmse is descriptive, not ranked)")
print(f"demand_target_frac (hashed factor): "
      f"{DtaliteODMEEstimator().factor_values['demand_target_frac']}")
assert odme_iters > 0  # never a 'Convergence reached after 0 iterations' no-op

ODME gradient iterations executed : 69  (>0: cleared the tol=1 floor)
od_identifiable                   : 0  (0 -> od_rmse is descriptive, not ranked)
demand_target_frac (hashed factor): 1.0

Anti-laundering — the certifier scores the OD, never DTALite’s self-report¶

The sacred property: the ONLY channel from odme-dtalite to the harness is the emitted OD matrix. ODCertifier re-runs its own bfw and never reads DTALite’s bookkeeping, so a buggy or adversarial run can at worst emit a bad OD that certifies honestly — it cannot forge a good certificate. DTALite reports a rosy count fit (its stalled-assignment, box-clamped predicted volumes); the certifier ignores it. That gap is the measured engine-in-the-loop bias, provenance only.

self_fit = float(bundle.trace.final.self_report["obs_count_rmse"])
certified = float(odme["obs_count_rmse"])
print(f"DTALite self-reported obs_count_rmse : {self_fit:.2f}  (its own optimistic view)")
print(f"pinned-bfw CERTIFIED obs_count_rmse  : {certified:.1f}  (what the leaderboard sees)")
print(f"engine-in-the-loop bias (self << certified): {self_fit < 0.5 * certified}")
assert self_fit < 0.5 * certified  # the certifier did not trust the self-report

DTALite self-reported obs_count_rmse : 8.06  (its own optimistic view)
pinned-bfw CERTIFIED obs_count_rmse  : 365.3  (what the leaderboard sees)
engine-in-the-loop bias (self << certified): True

fig, ax = plt.subplots(figsize=(5.2, 3.2))
labels = ["obs_count_rmse", "heldout_count_rmse\n(ranking)"]
prior_vals = [float(prior["obs_count_rmse"]), float(prior["heldout_count_rmse"])]
odme_vals = [float(odme["obs_count_rmse"]), float(odme["heldout_count_rmse"])]
x = range(len(labels))
ax.bar([i - 0.2 for i in x], prior_vals, width=0.4, label="prior baseline")
ax.bar([i + 0.2 for i in x], odme_vals, width=0.4, label="odme-dtalite")
ax.set_xticks(list(x)); ax.set_xticklabels(labels)
ax.set_ylabel("certified count RMSE"); ax.set_title("odme-dtalite vs prior (pinned-bfw certified)")
ax.legend(); fig.tight_layout()

../../_images/8acfae6ea1c1d8e02192f9c6a69edbdff2bea1429038c9eb51d02e8d7c7ba231.png

Takeaways¶

odme-dtalite is a guarded static estimator on the UNCHANGED pinned-bfw certifier — the adr-028 zero-certifier-change ideal, a second engine after spsa-sumo.
It measurably beats the prior on the certified count fit at the Sioux Falls marquee, with the ODME descent actually firing (>0 iterations) — the anchor is chosen to clear DTALite’s hardcoded tol=1 floor.
Its engine envelope (the [0.5,1.5]× box, the hardcoded weights/tol, the route_output=1 requirement, the link_performance.csv corruption it routes around by reading od_performance.csv only) is disclosed as an honest scope boundary. Single-mode demand-only; multiclass ODME is a deferred follow-up.

See docs/design/adr-042-odme-dtalite.md.