learned-surrogate — A learned UE surrogate, and how the harness CENSORS it

What. learned-surrogate is a numpy ridge regression trained to predict link volume/capacity ratios directly — no equilibrium is solved at inference. Because it regresses each link independently, its emitted flows need not conserve demand, and this notebook shows what the harness does about that ([rahman2023data], docs/REFERENCES.md).

Why it is in the benchmark. It is the honest-censoring case (ADR-026): a model that emits demand-infeasible flows is CENSORED — its gap is nan, not a score — so garbage can neither crash the experiment nor top the leaderboard (crash-vs-censor, P1). See the model compendium and docs/ARCHITECTURE.md.

Scope. Runs on the built-in Braess scenario and certifies that the surrogate’s raw output is censored. This is a NEGATIVE result on purpose — the censoring IS the lesson.

How this notebook is graded

A notebook never claims a number it does not compute in that cell. Every scored quantity below is recomputed live by the P1 Evaluator from the flows the model emitted, in the cell where it is claimed. Model self-reports are shown only as provenance and diffed against the certificate as an honesty check, exactly as the harness treats them (README, Certified, not self-reported).

# Setup. `learned-surrogate` is a core model: a plain `pip install -e .` suffices — no
# optional extra, so no guard cell. The inline backend is Agg-based (headless CI
# renders into the notebook); NEVER matplotlib.use("Agg") in-kernel — it silently
# suppresses inline figure capture.
%matplotlib inline
import numpy as np

from tabench import (
    Budget,
    Evaluator,
    LearnedSurrogateModel,
    RngBundle,
    Trace,
    braess_scenario,
    viz,
)

The scenario

The built-in Braess network: 4 nodes, 5 links, a single OD pair (1 → 2) with demand 6. Scenarios are frozen and content-hashed (P2) — the hash printed below is the identity of the benchmark instance, so a silently edited network cannot masquerade as it.

scenario = braess_scenario()
net = scenario.network

print(f"scenario      : {scenario.name}")
print(f"content hash  : {scenario.content_hash()[:16]}…")
print(f"links         : {net.n_links}  (tail→head: "
      + ", ".join(f"{i}->{j}" for i, j in zip(net.init_node, net.term_node)) + ")")
print(f"total demand  : {scenario.demand.total}")

scenario      : braess
content hash  : cf00f411cdccec88…
links         : 5  (tail→head: 1->3, 1->4, 3->4, 3->2, 4->2)
total demand  : 6.0

Solve

The model contract (CONTRIBUTING.md): a model receives (scenario, budget, rng, trace), records checkpoints, and respects the budget. Budgets are hardware-free (iterations / shortest-path calls; wall-clock is recorded but never the ranking axis, P7). Whatever the model writes into self_report is provenance, not a score.

model = LearnedSurrogateModel()
bundle = model.solve(scenario, Budget(iterations=1), RngBundle(0), Trace())

final = bundle.final
print(f"model              : {model.name}")
print(f"emitted flows      : {np.round(final.link_flows, 4)}")
# The surrogate self-reports a predicted mean v/c ratio, but NO gap: it does not solve
# an equilibrium, so `provides_gap=False` and there is no self-gap to trust or diff.
print(f"self-report keys   : {sorted(final.self_report)}  (no relative_gap)")

model              : learned-surrogate
emitted flows      : [4.3202 0.0143 4.3202 0.0143 4.3202]
self-report keys   : ['predicted_mean_vc', 'training_sp_calls', 'training_wall_ms']  (no relative_gap)

Certify (P1) — the censoring

The harness scores the surrogate with the same certificate as every solver. Its raw flows fail the demand-feasibility audit (node balance ≠ 0), so feasible = 0 and the relative gap is censored to nan rather than scored. We recompute the true UE in- cell to show how far off the raw flows are — the censored gap is not hidden, it is reported as nan.

evaluator = Evaluator(scenario)
metrics = evaluator.evaluate(final.link_flows)
print(f"feasible               : {metrics['feasible']:.0f}   (0 = censored)")
print(f"certified relative gap : {metrics['relative_gap']}   (nan = censored, not scored)")
print(f"node balance residual  : {metrics['node_balance_residual']:.4f}   (demand not conserved)")

# The whole point: a demand-infeasible learned output is CENSORED, not scored.
assert metrics["feasible"] == 0.0
assert np.isnan(metrics["relative_gap"])
assert metrics["node_balance_residual"] > 1e-6

# No honesty diff: the surrogate self-reports no gap (provides_gap=False).
assert "relative_gap" not in final.self_report

# Analytic anchor RECOMPUTED: the true Braess UE, which the raw surrogate is far from.
ref_flows = np.array([4.0, 2.0, 2.0, 2.0, 4.0])
assert evaluator.evaluate(ref_flows)["relative_gap"] < 1e-6
assert not np.allclose(final.link_flows, ref_flows, atol=1e-1)
print(f"true UE flows          : {ref_flows}  (feasible, gap ~ 0)")

feasible               : 0   (0 = censored)
certified relative gap : nan   (nan = censored, not scored)
node balance residual  : 1.6655   (demand not conserved)
true UE flows          : [4. 2. 2. 2. 4.]  (feasible, gap ~ 0)

Visualize

Both figures come from tabench.viz. Left/top: the surrogate’s raw link flows on the Braess diamond — visibly not a conserved routing. Right/bottom: the raw flows against the true UE; the points sit far OFF the y = x guide, the censored gap made visual. (For a learned model that IS feasible by construction, see the torch implicit-ue-nn tutorial.)

display(viz.plot_network_flows(net, final.link_flows))
display(viz.plot_flow_scatter(("analytic UE", ref_flows), {"learned-surrogate": final.link_flows}))

Takeaways & pointers

• Censored, not crashed, not scored. Demand-infeasible flows get feasible = 0 and a nan gap — garbage neither breaks the run nor tops the leaderboard (P1).

• No self-gap to trust. The surrogate solves no equilibrium, so it reports none; the harness is the only authority.

• Where next. A learned model feasible BY CONSTRUCTION: the torch implicit-ue-nn tutorial (planned, 10-learned/, batch-11 — see tutorials/README.md’s index; no link yet, it hasn’t shipped); the certified solvers: bfw; ADR-026 in the model compendium.