tabench.core

The object model: the frozen, content-hashed Scenario, the Capabilities contract, Budget, the Trace, and the RNG discipline every model runs under.

Core abstractions: scenarios, capabilities, budgets, traces, randomness.

class tabench.core.Budget(iterations=None, sp_calls=None, wall_seconds=None, target_relative_gap=None)

Bases: object

Resource limits for one solve. None means unconstrained on that axis.

target_relative_gap is an optional convergence early-stop, not a resource axis: solvers that monitor their own convergence measure MAY stop once it is reached (the certified metrics are still recomputed by the harness). For UE solvers the measure is the relative gap; for the SUE solver it is the fixed-point residual (ADR-001). Boyce, Ralevic-Dekic & Bar-Gera (2004) recommend RG <= 1e-4 for stable link flows; a resource axis is still required so that non-converging models (and single-shot models, which ignore the target) terminate.

Parameters:

• iterations (int | None)

• sp_calls (int | None)

• wall_seconds (float | None)

• target_relative_gap (float | None)

target_met(relative_gap)

True when a self-monitored relative gap satisfies the target.

Parameters:

relative_gap (float | None)

Return type:

bool

class tabench.core.BudgetCoords(iterations=0, sp_calls=0, wall_ms=0.0)

Bases: object

Where a checkpoint sits in budget space. All coordinates recorded (P6).

Parameters:

• iterations (int)

• sp_calls (int)

• wall_ms (float)

class tabench.core.Capabilities(paradigm, deterministic, provides_gap, seedable, inputs_required=frozenset({'od_matrix'}), outputs=frozenset({'link_flows'}), trained_on=())

Bases: object

What a model declares about itself; the harness trusts nothing else.

provides_gap means the model self-certifies an equilibrium gap. The harness still recomputes every scored metric externally (P1); self-reports are only diffed against harness values as an honesty check.

trained_on lists scenario families in the model’s training lineage (learned models). The fairness gate refuses evaluation on scenarios whose family appears here.

Parameters:

• paradigm (str)

• deterministic (bool)

• provides_gap (bool)

• seedable (bool)

• inputs_required (frozenset[str])

• outputs (frozenset[str])

• trained_on (tuple[str, ...])

exception tabench.core.ContaminationError

Bases: RuntimeError

Raised when a model’s training lineage intersects the evaluation scenario.

tabench.core.assert_fair_evaluation(capabilities, scenario)

Fairness gate (P7): refuse train/test contamination.

Declared lineage tokens are matched against both the scenario’s family name and its content hash. Declaring content hashes is robust to scenario renaming; declaring only family names is not (a byte-identical scenario republished under a new family name will not match). Learned models should therefore declare content hashes of their training scenarios whenever available, and family names as a coarser fallback.

Parameters:

• capabilities (Capabilities)

• scenario (Scenario)

Return type:

None

class tabench.core.FactorSpec(default, kind='float', bounds=None, doc='')

Bases: object

Specification of one model hyperparameter.

Parameters:

• default (Any)

• kind (str)

• bounds (tuple[float, float] | None)

• doc (str)

tabench.core.resolve_factors(specs, overrides)

Merge user overrides into declared factor defaults, validating names/bounds.

Parameters:

• specs (dict[str, FactorSpec])

• overrides (dict[str, Any])

Return type:

dict[str, Any]

class tabench.core.FlowState(link_flows, coords, self_report=<factory>, class_link_flows=None)

Bases: object

One emitted solution state.

class_link_flows is an OPTIONAL (n_classes, n_links) per-class flow matrix (default None). Single-class models never set it, so every shipped model and the aggregate link_flows contract are byte-identical to before this field existed. Multiclass models (Dafermos 1972, adr-013) emit it as a first-class object — the harness recomputes the per-class VI residual from it (P1), so it is not a self-report. When present, link_flows is the class sum class_link_flows.sum(axis=0).

Parameters:

• link_flows (ndarray)

• coords (BudgetCoords)

• self_report (dict[str, float])

• class_link_flows (ndarray | None)

class tabench.core.ResultBundle(model_name, final, trace, factors=<factory>, seed_info=<factory>)

Bases: object

Everything one solve produced, with provenance.

Parameters:

• model_name (str)

• final (FlowState)

• trace (Trace)

• factors (dict[str, Any])

• seed_info (dict[str, Any])

class tabench.core.Trace

Bases: object

Ordered stream of checkpoints emitted during one solve.

record(link_flows, coords, class_link_flows=None, **self_report)

Record a checkpoint. Flows are copied defensively.

class_link_flows (optional, multiclass only) is copied and stored for the harness’s per-class certificate; None for every single-class model, leaving their emissions unchanged.

Parameters:

• link_flows (ndarray)

• coords (BudgetCoords)

• class_link_flows (ndarray | None)

• self_report (float)

Return type:

None

class tabench.core.RngBundle(root_seed, macrorep=0)

Bases: object

Factory of independent generators keyed by (macrorep, source, replication).

Parameters:

• root_seed (int)

• macrorep (int)

generator(source, replication=0)

Return the deterministic generator for one randomness source.

The same (root_seed, macrorep, source, replication) always yields an identical stream; distinct tuples yield independent streams.

Parameters:

• source (int)

• replication (int)

Return type:

Generator

describe()

Provenance snapshot for manifests.

Return type:

dict[str, int]

class tabench.core.Demand(matrix)

Bases: object

Origin-destination demand. matrix[i, j] is flow from zone i+1 to j+1.

Parameters:

matrix (ndarray)

class tabench.core.ElasticDemand(form, param)

Bases: object

Elastic (variable) demand law: the demand between an OD pair is a strictly decreasing function of that pair’s equilibrium travel cost, d_rs = D_rs(u_rs).

The reference demand d0_rs = D_rs(0) (demand at zero cost — an upper bound) is carried by the scenario’s ordinary Demand matrix; this object adds only the decay law and its single shared parameter. Two forms:

• "linear" D_rs(u) = d0_rs * max(0, 1 - u/param) (param = u0) inverse on [0, d0]: D_rs^{-1}(d) = param * (1 - d/d0) — bounded, singularity-free (the excess-demand arc cost param * e/d0 is linear).

• "exponential" D_rs(u) = d0_rs * exp(-param * u) (param = beta) inverse on (0, d0]: D_rs^{-1}(d) = ln(d0/d) / param — the cited/logit-consistent form, unbounded as d -> 0 (guarded below).

Objective min Sum_a integral t_a - Sum_rs integral D_rs^{-1} and the excess-demand (“dummy arc”) transformation follow Sheffi (1985) ch. 6 and Boyles, Lownes & Unnikrishnan, Transportation Network Analysis sec. 9.1; the transformation itself is due to Gartner (1980), and the seminal computational treatment is Florian & Nguyen (1974). See docs/design/adr-005.

param is a scalar shared across OD pairs (per-OD curves are a future extension). It is task data, content-hashed with the scenario when set.

Parameters:

• form (str)

• param (float)

realized_demand(d0, cost)

D_rs(u) elementwise: realized demand at OD cost cost, clamped to [0, d0]. Both arguments broadcast (typically (n_zones, n_zones) reference-demand and OD shortest-path-cost matrices).

Parameters:

• d0 (ndarray)

• cost (ndarray)

Return type:

ndarray

inverse_demand(d0, d)

D_rs^{-1}(d): the OD cost at which realized demand equals d (d in [0, d0]). Prices the excess-demand arc. For the exponential form the d -> 0 log-singularity is floored so the transformed solver never evaluates ln(inf); d0 == 0 pairs (no demand) return 0.

Parameters:

• d0 (ndarray)

• d (ndarray)

Return type:

ndarray

excess_arc_cost(d0, excess)

Cost of the excess-demand (Gartner dummy) arc carrying unmet flow excess: W(e) = D_rs^{-1}(d0 - e) — increasing in e.

Parameters:

• d0 (ndarray)

• excess (ndarray)

Return type:

ndarray

class tabench.core.CombinedDemand(productions, attractions, beta)

Bases: object

Combined trip-distribution + assignment task data (Evans 1976).

Instead of a fixed OD matrix, the demand is endogenous: only the trip-end margins are fixed — the productions O_i (trips originating in zone i) and attractions D_j (trips destined for zone j) — and the OD matrix d_ij is distributed across pairs by a doubly-constrained gravity model at the equilibrium travel costs. The combined convex program (Evans 1976; Sheffi, Urban Transportation Networks 1985 ch. 6; Boyles, Lownes & Unnikrishnan, Transportation Network Analysis §6) is

min_{x,d} Σ_a ∫_0^{x_a} t_a(w) dw + (1/β) Σ_ij d_ij (ln d_ij − 1) s.t. Σ_j d_ij = O_i, Σ_i d_ij = D_j, d ≥ 0, x = assign(d)

whose stationarity conditions give the doubly-constrained gravity d_ij = A_i B_j exp(−β u_ij) (u_ij the equilibrium OD cost, A/B the balancing factors) and Wardrop route equilibrium of d. The single parameter beta is the gravity dispersion / impedance sensitivity (large beta → trips concentrate on cheap destinations; beta → 0 → the cost-free maximum-entropy distribution O_i D_j / T).

Intrazonal trips never enter the network, so the gravity is distributed over interzonal pairs only (i ≠ j with O_i > 0 and D_j > 0). This is task data, content-hashed with the scenario (docs/design/adr-007); the reference Demand matrix carries the free-flow gravity distribution (a deterministic reference — the uncongested equilibrium demand). See docs/design/adr-007-combined-distribution-assignment.md.

Parameters:

• productions (ndarray)

• attractions (ndarray)

• beta (float)

support()

Boolean (n_zones, n_zones) mask of distributable OD pairs: interzonal (i ≠ j) pairs with positive production and attraction.

Return type:

ndarray

gravity(od_cost, *, tol=1e-13, max_iters=10000)

Doubly-constrained gravity demand d_ij = A_i B_j exp(−β u_ij) at OD costs u = od_cost (a (n_zones, n_zones) matrix; only support entries are read), balanced to the productions/attractions by the classic Furness / iterative-proportional-fitting recursion

A_i = O_i / Σ_j B_j f_ij, B_j = D_j / Σ_i A_i f_ij, f_ij = exp(−β u_ij) on the interzonal support (0 elsewhere).

Deterministic (fixed tolerance/iteration cap) so the harness certificate and the solver recompute byte-identical demand from the same costs (P1). A final row rescale makes the production margins exact; the attraction margins converge to tol. Returns a dense matrix, zero off support.

Parameters:

• od_cost (ndarray)

• tol (float)

• max_iters (int)

Return type:

ndarray

class tabench.core.MulticlassDemand(matrices, interaction)

Bases: object

Multiclass-user demand + linear class-interaction (Dafermos 1972; adr-013).

K >= 2 user classes share one physical network. Each class i has its own OD demand matrix matrices[i] (so the classes couple only through the link costs, never the demand constraints — the feasible set is a Cartesian product of per-class demand polytopes) and perceives a class-specific link cost that is the shared BPR latency of the total link flow plus a linear class interaction

t_a^i(V) = t_a^BPR(v_a) + sum_j interaction[i, j] * v_a^j, v_a = sum_j v_a^j (total flow on link a), v_a^j = class j’s flow on a.

interaction is a (K, K) matrix applied per link (units of cost per unit class-j link flow). It is what makes the split among classes well-defined: with interaction = 0 every class sees the identical cost t_a^BPR(v_a) and the per-class split is arbitrary (the model degenerates to a single-class UE on the summed demand). A symmetric interaction (interaction[i,j] == interaction[j,i]) makes the equilibrium the minimizer of a convex multiclass-Beckmann potential (the integrable case Dafermos 1972 characterizes); an asymmetric one makes it a genuine variational inequality with no equivalent optimization (Smith 1979; Dafermos 1980) — the block-structured generalization of the single-class vi-asym cost t(v) = t_BPR(v) + C v.

This is task data, content-hashed with the scenario when set. No sign constraint is imposed on interaction (a pathological entry that drives a cost non-positive is caught by the solver and censored by the certificate, exactly as for vi-asym).

Parameters:

• matrices (ndarray)

• interaction (ndarray)

property total_matrix: ndarray

Class-summed OD matrix — the aggregate demand the network carries.

symmetric(tol=1e-12)

True iff the class interaction is symmetric (the integrable case).

Parameters:

tol (float)

Return type:

bool

class tabench.core.Network(name, n_nodes, n_zones, first_thru_node, init_node, term_node, capacity, length, free_flow_time, b, power, toll, link_type, toll_weight=0.0, distance_weight=0.0, units=())

Bases: object

Directed road network with BPR-family link performance functions.

The generalized link cost (in time units) is

t_a(v) = fft_a * (1 + b_a * (v / cap_a)^power_a)

• toll_weight * toll_a + distance_weight * length_a

where the toll and distance weights are per-network metadata (they are NOT stored in .tntp files; see docs/ARCHITECTURE.md P9).

units records per-network unit conventions (e.g. Sioux Falls free-flow times are 0.01 hours) as documentation; costs are always computed in the network’s native units.

Parameters:

• name (str)

• n_nodes (int)

• n_zones (int)

• first_thru_node (int)

• init_node (ndarray)

• term_node (ndarray)

• capacity (ndarray)

• length (ndarray)

• free_flow_time (ndarray)

• b (ndarray)

• power (ndarray)

• toll (ndarray)

• link_type (ndarray)

• toll_weight (float)

• distance_weight (float)

• units (tuple[tuple[str, str], ...])

property fixed_cost: ndarray

Flow-independent generalized-cost component per link.

link_cost(link_flows)

Generalized link travel cost t_a(v_a) at the given flows.

Parameters:

link_flows (ndarray)

Return type:

ndarray

link_cost_integral(link_flows)

Per-link Beckmann integral: integral of t_a(s) ds from 0 to v_a.

Parameters:

link_flows (ndarray)

Return type:

ndarray

link_cost_derivative(link_flows)

Per-link derivative dt_a/dv_a — the diagonal Hessian of the Beckmann objective, used by conjugate-direction Frank-Wolfe variants.

For BPR, t’(v) = fft * b * p * (v/cap)^(p-1) / cap. Edge cases: p = 0 or b = 0 gives exactly 0 (constant cost); p = 1 gives the constant fft*b/cap (numpy 0**0 = 1); 0 < p < 1 has an unbounded derivative at v = 0, where H is defined as 0 (it is only a conjugacy scale) and 0**(negative) is never evaluated.

Parameters:

link_flows (ndarray)

Return type:

ndarray

class tabench.core.ReferenceSolution(link_flows, source, note='')

Bases: object

Best-known solution used as a regression oracle (with provenance).

Parameters:

• link_flows (ndarray)

• source (str)

• note (str)

class tabench.core.Scenario(name, network, demand, reference=None, family='', sue_theta=None, sue_family='logit', elastic_demand=None, combined_demand=None, br_epsilon=None, side_capacities=None, link_interaction=None, multiclass=None)

Bases: object

A frozen benchmark instance: network + demand (+ optional oracle).

family names the scenario lineage used by the fairness gate (P7): a learned model whose trained_on includes this family is refused evaluation on this scenario.

sue_theta (optional) makes this an SUE task: the dispersion dial of the pinned loading map, in 1/(native cost unit) — so its meaning is network-specific (P9; scenario cards state the unit). It is task data, never a model factor, and it is content-hashed when set (two scenarios differing only in theta are different benchmark instances).

sue_family selects the choice-model family of that SUE task — "logit" (Dial-STOCH closed-form loading, docs/design/adr-001) or "probit" (Monte Carlo loading + pinned certificate, adr-003). For probit, sue_theta carries beta, the perception variance per unit free-flow time. This is unrelated to family, which is data lineage for the trained_on fairness gate (P7); sue_family names the task’s equilibrium definition. It is hashed only when non-default so every logit scenario keeps the byte-identical hash it had before this field existed.

elastic_demand (optional) makes this a variable-demand UE task: the Demand matrix becomes the reference demand d0 = D(0) and this ElasticDemand supplies the decay law (docs/design/adr-005). Like sue_theta it is task data content-hashed only when set, so every fixed-demand scenario keeps its prior hash. An elastic task is a deterministic UE with endogenous demand and so is mutually exclusive with the SUE fields.

combined_demand (optional) makes this a combined trip-distribution + assignment task (Evans 1976; docs/design/adr-007): the OD matrix is endogenous, distributed by a doubly-constrained gravity model from the fixed trip-end margins in this CombinedDemand, and the Demand matrix carries the free-flow gravity reference. Like the other optional task fields it is content-hashed only when set. It is mutually exclusive with both the SUE fields and elastic_demand (all three make demand non-fixed in incompatible ways).

br_epsilon (optional) makes this a boundedly-rational user-equilibrium task (Mahmassani & Chang 1976/1987; docs/design/adr-008): a flow is acceptable if every used route lies within an absolute indifference band epsilon (native cost units) of its OD’s minimum route cost, c_pi <= kappa_rs + epsilon. This relaxes Wardrop’s equality to a one-sided band, so the equilibrium is a set, not a point (epsilon=0 recovers UE). It is task data, content-hashed only when set, and mutually exclusive with the SUE / elastic / combined fields (BR-UE is a deterministic fixed-demand route equilibrium; those make the demand non-fixed). The band unit is network-specific (scenario cards state it, P9).

side_capacities (optional) makes this a side-constrained UE task (Larsson & Patriksson 1995; docs/design/adr-009): a per-link hard capacity u_a on the link flow, v_a <= u_a, imposed in addition to the BPR latency (these are physical throughput limits, distinct from the BPR Network.capacity reference volume). The equilibrium is Wardrop UE on the capacity-augmented cost t_a(v_a) + beta_a with beta_a >= 0 a queueing delay / toll that is zero off the binding set. It is per-link task data, content-hashed only when set, and mutually exclusive with the SUE / elastic / combined / BR fields.

multiclass (optional) makes this a multiclass-user equilibrium task (Dafermos 1972; docs/design/adr-013): K >= 2 user classes share the network, each with its own OD demand and a class-specific link cost coupling the classes (MulticlassDemand). The aggregate demand must equal the class sum (so single-class consumers of demand stay consistent). It is task data, content-hashed only when set, and mutually exclusive with the SUE / elastic / combined / BR / side-constrained / link-interaction fields (link_interaction is its single-class special case). Certifying it needs the model’s per-class link flows (FlowState.class_link_flows); the scored quantity is the class-summed VI residual.

Parameters:

• name (str)

• network (Network)

• demand (Demand)

• reference (ReferenceSolution | None)

• family (str)

• sue_theta (float | None)

• sue_family (str)

• elastic_demand (ElasticDemand | None)

• combined_demand (CombinedDemand | None)

• br_epsilon (float | None)

• side_capacities (ndarray | None)

• link_interaction (ndarray | None)

• multiclass (MulticlassDemand | None)

content_hash()

SHA-256 over the canonical serialization of all scored content (P2).

Return type:

str