Collective / gated regimes + edge-fold of IID shocks

[hmgaudecker]

What this PR adds

This PR lets pylcm represent collective life-cycle decisions in which several
people choose one household action but retain distinct utilities, continuation
values, and outside options. It also adds two complementary ways to handle
uncertainty without unnecessarily enlarging stored value functions: folding a
purely transitory IID shock and declaring a correlated JointTransition.

The existing singleton-regime workflow remains the default.

Economic model

Collective choice

A regime becomes collective by declaring a CollectiveUtility in the slot it
already has for utility. For each feasible state-action cell, pylcm constructs
one action value per stakeholder, Q_s(x, a). The household chooses one common
action

a*(x) = argmax_a sum_s lambda_s Q_s(x, a),

and stores every stakeholder's value Q_s(x, a*(x)) at that same choice. This
is the key distinction from solving separate optimization problems for the two
partners.

Everything is declared in a slot the regime already has, so there is exactly one
way to write a collective regime:

Regime(
    functions={
        "utility": CollectiveUtility(
            utilities={"f": u_f, "m": u_m},
            objective=ParetoObjective(weights={"f": power}),  # omit for equal weights
        )
    },
    constraints={
        "participation_f": ValueDependentConstraint(
            predicate=participation_f,
            references={
                "V_alone_f": ProjectedRegimeValue(
                    regime="single_f", projection={"wealth": half_of_wealth}
                )
            },
        )
    },
    ...
)

The utilities keys are the stakeholders, in insertion order, and that order
fixes the trailing axis of V and of every published array. A stakeholder whose
body is None delegates to a utility_<s> entry arriving from the model level.

Value constraints may read each stakeholder's Q_s, which is what supports
limited-commitment applications where household decisions must satisfy
individual rationality rather than only maximize a household objective. When no
action satisfies all participation constraints, the cell carries a dissolution
flag; solve(..., return_dissolution_flags=True) publishes those flags and
simulate(period_to_regime_to_dissolution_flags=...) takes them back.

Consent, separation, and fallback regimes

ValueDependentTransition declares a transition whose continuation depends on a
Boolean economic condition — mutual consent before entering a couple regime, or
dissolution when a participation constraint fails. It goes in the transition
slot, keyed by target:

transition={
    "couple": ValueDependentTransition(
        probability=MarkovTransition(stays_married),
        gate=no_dissolution,
        routes={"f": StakeholderRoute(target_stakeholder="f", fallback=alone_f)},
        gate_references={
            "V_alone_f": ProjectedRegimeValue(
                regime="single_f", projection={"wealth": half_of_wealth}
            )
        },
        off_grid="pointwise",
    )
}

The key is always the gate-open target. A dissolution edge is keyed by the
continuing collective regime under gate = ~D_target; keying it by the
singleton would send both partners there whenever the couple stays together.

routes is keyed by source stakeholder, and each route owns four destinations:
the open regime (the dict key) and role (target_stakeholder), and the closed
regime and role (fallback.regime, fallback.stakeholder). If the gate is open
the leg reads the appropriate stakeholder value in the target regime; if closed
it reads the declared same-period fallback, usually that stakeholder's single
regime. The ordinary regime-transition draw still determines which edge is
attempted. Solve and simulation use the same routing declarations, including
target-specific state projections and stakeholder roles.

Simulation follows a fixed-size cohort of rows. A separation does not create two
new rows; own_stakeholder says which stakeholder each simulated row follows
after stakeholder-specific routing.

One declaration surface

An earlier revision of this branch shipped two public spellings for the same
regime: the declarations above, and the lowered form the engine runs after
pylcm takes them apart — stakeholders, pareto_objective, value_constraints,
same_period_refs, gated_edges as Regime arguments, plus a public
GatedEdge class.

Publishing both would have made the equivalence and collision rules between two
spellings part of the long-term public API. The lowered vocabulary has been
removed from the public surface:

• the five slots are now field(init=False) derived values. They survive as
  outputs of lowering, which is the supported way to ask what a declaration
  produced, but they cannot be passed to the constructor or to replace;
• lowering no longer consumes functions, constraints and transition, so the
  declaration objects stay where the author wrote them, and the engine reads
  three derived views instead (decomposed_functions, decomposed_constraints,
  decomposed_transition);
• GatedEdge moved to _lcm.gated_edge and left lcm.__all__.

This costs no user. An AST walk over every *.py under tests/ — counting
ast.keyword nodes named after one of the five slots, 0 parse failures at each
ref — finds 0 occurrences at the branch point and 7 at the head, none of
them to Regime(...). The lowered vocabulary never existed before this branch,
so removing it before release breaks no promise that was ever made. The seven
survivors are engine vocabulary — five _MockRegime (which bypasses __init__),
one build_pareto_weights, one dataclasses.replace on a canonical engine
regime — and each is pinned by name in a census test that fails if an unlisted
one appears.

Closing the lowered form also required widening the declarations to cover three
configurations that had been reachable only in lowered form: a gated edge inside
a Phased transition, a Phased per-stakeholder utility, and per-stakeholder
utility bodies arriving from the model level.

Uncertainty and value-function size

Folded IID shocks

An IID process can set fold=True when its realization is observed, used in the
current decision, and then discarded. pylcm still solves the choice problem at
every quadrature node and then averages the resulting values,
E_epsilon[max_a Q(x, epsilon, a)], but does not store the shock as a value-
function axis.

This is deliberately restricted to cases where the reduction is exact: a fully
specified, non-persistent IID process under linear expectation, solved by grid
search, with no later transition or same-period value read depending on the
discarded realization. A folded shock whose conditioning state can move between
periods is refused, because the fold would otherwise price a distribution the
declared transition does not use. Unsupported combinations fail during model
construction.

Correlated next-state innovations with JointTransition

Separate MarkovTransition declarations imply separate draws. That is wrong
when one innovation jointly determines several next-period states—for example,
a partner offer carrying correlated earnings and health characteristics.

JointTransition declares one finite-support lottery on a particular
source-target edge and several output laws that share the same support row.
Backward induction forms the expectation over that common draw inside the
source action value before maximizing, while simulation samples one row and
uses it for every output. The latent realization is transition-local: it is not
a model state, initial condition, simulation column, or stored value-function
axis. Phased can give solving and simulation different perceived and realized
joint laws while preserving the same structural support.

An output may land on a stochastic process's own nodes as well as an ordinary
grid, which is how correlated innovations reach a grid pylcm discretized rather
than one discretized by hand. Probability vectors are never silently normalized:
a lottery that does not return one probability vector per source cell is an
error, raised in the params-bound preflight.

Transition architecture

The branch removes the former TransitionLaws metadata representation. Ordinary
deterministic and Markov state laws, stochastic-process laws, explicit process
entry, interpolation bases, and JointTransition kernels now lower to one
target-specific TargetTransitionPlan. Solve, simulation, validation,
diagnostics, and solver-scope checks consume that same plan.

Regime-transition probabilities remain a separate concern: they select the
target regime, whereas a target transition plan determines the target states
conditional on that target. Both public grammars are canonicalized by reachable
target, but combining target selection and conditional state evolution into one
public object would blur rather than remove this distinction. This supersedes
the internal-representation question raised in #423 without adding a wrapper
around Regime.transition.

Correctness and supported scope

The implementation also adds the less visible machinery needed to make these
features compose safely:

• solve/simulation phase closure, target-period and age-specialized-grid
  handling, and target-specific parameter namespaces;
• explicit ownership of transition outputs and rejection of overlapping or
  incomplete producers;
• zero-mass and extended-real-safe arithmetic, avoiding 0 * -inf -> NaN in
  regime mixtures, stakeholder aggregation, and gated fallbacks;
• Pareto weights that stay representable under normalization, and that are
  judged even when a weight reads a carried state, whose solve value comes from
  its imputation rather than from a grid axis;
• deterministic routing under ahead-of-time compilation and batched
  simulation; and
• fail-closed checks for solver or feature combinations whose economic timing
  is not implemented.

Collective regimes currently use GridSearch and linear expectation and do not
combine with folded shocks or EV1 taste shocks. Transition-local
JointTransition lotteries likewise use GridSearch; EGM-family solvers reject
them explicitly. A gated edge on a coarse or terminal transition is unsupported
and refused: a gate is a route, and a transition naming no target has no route
to fall back to.

The branch includes end-to-end tests of the documented public workflow and
class-level regression coverage for collective choice, participation and
dissolution, gated routing, folded-shock timing, joint-transition correlation,
parameter ownership, float32/float64 edge cases, and solve/simulation/AOT
parity.

Review status

Nine bounded external mathematical/code-review rounds are complete. The definitive
Round-9 closure audit returned CLEAN / CLOSED: F1-F9 are verified_fixed, both
required profiles pass, no finding remains open, and next_action is done.

The final candidate certificate is an exact route-local direct-data-flow proof over all
six GridSearch corridors (ordinary and taste-shock singleton/collective solve and
simulation). It seals the 43-source dependency frontier and proves that the exact Q_arr
and F_arr produced by Q_and_F reach the full reducer and publication path without a
candidate-changing transformation. The synchronized closure suites reject all 66/66
semantic mutations and all 176/176 full re-anchored mutations, including the historical
MT9 and MT10 escapes.

The follow-up for review 5064008752 is complete at 90fe284f. It preserves ordinary
DAG ancestors when carried-state Pareto weights are folded into solve-time imputation,
retains narrow reachability for Phased per-target transitions, and updates the public
Model docstrings to name ValueDependentTransition rather than the lowered internal
edge type. This repair head also incorporates #407 at 1c9f1d22 and re-anchors the source
inventory and direct-flow seals to the combined production tree. After #407 landed on
main at b803c810, the tree-preserving post-land merge produced current head
723f9a26; the inventory and unified verifier remain green. PR #409 was then
squash-merged into main at e7e4a83f.

Focused verification on the combined topology is green: float32 has 193 passed and 5
skipped; float64 has 198 passed; neither profile has failures or errors. The unified
static verifier passes, and the synchronized self-test rejects all 176/176 direct-flow
mutations.

Round-9 closure evidence at 1bff3e16:

• candidate certificate: 93/93 passed at float32 and 93/93 at float64;
• full float32 profile: 6,787 passed, 316 skipped, 7 xfailed, 0 failed/errors;
• full float64 profile: 7,083 passed, 55 skipped, 7 xfailed, 0 failed/errors;
• prek run --all-files: all 34 hooks passed without modifying files.

Round 9 changes only the four certificate/test paths; no production src/ path changes.
The closure state passes the strict audit gate with zero warnings or blockers.

A Windows portability follow-up at 3272af34 makes the sealed-source hashes
independent of checkout newline convention. The certificate now canonicalizes UTF-8
text before hashing, so CRLF checkouts match the LF inventory while any non-newline
source change remains detectable. The expanded candidate-certificate battery passes
95/95 at both float32 and float64; inventory generation, the unified contract/policy
verifier, all 176 synchronized mutations, and an actual core.autocrlf=true clone also
pass. This follow-up changes three test/tooling paths and no production src/ path.

Stack and related issues

This PR was stacked on feat/continuous-outer and merged after #407. The
collective, gated-edge, folded-shock, and transition-plan work touches the same
Bellman, transition, parameter, and simulation boundaries, so it is kept on the
single audited branch rather than split after the fact.

• Closes Dissolution routing: revert each individual to its own single regime (synthetic mode) #401 (stakeholder-specific dissolution routing).
• Closes Fold IID utility shocks out of the stored value (quadrature-average at solve) #402 (fold IID utility shocks out of stored values).
• Supersedes the representation question in Unify regime / state transitions? #423; see Transition
  architecture above.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[read-the-docs-community]

Documentation build overview

📚 pylcm | 🛠️ Build #34309834 | 📁 Comparing 723f9a2 against latest (f03c1c9)

  🔍 Preview build  

46 files changed · + 9 added · ± 37 modified

+ Added

• adding-a-solver/index.html
• age-specialized/index.html
• choosing-a-solver/index.html
• collective-regimes/index.html
• collective-resource-contract/index.html
• continuous-integration/index.html
• credits/index.html
• epstein-zin/index.html
• nbegm/index.html

± Modified

• index.html
• approximating-continuous-shocks/index.html
• architecture/index.html
• benchmarking/index.html
• benchmarking-1/index.html
• beta-delta/index.html
• continuous-stochastic-processes/index.html
• conventions/index.html
• debugging/index.html
• defining-models/index.html
• and 27 more...

[hmgaudecker]

Head-vs-base failure diff — this branch causes 8 failures, all in tests/test_ndimage.py

collective-regimes @ 5c0b829: 76 failures. feat/continuous-outer @ db00f64: 68.
Caused by this PR: 8. Inherited: 68. Fixed by this PR: 0.

CI could not answer this. main.yml runs pixi run -e tests-cpu tests with no -rf, so
no failure is ever named — the ubuntu job printed 53 F marks and zero test names — and
that job additionally dies at ~96% with "the runner has received a shutdown signal". So
this was measured locally, chunked, one fresh process per chunk.

Caused by this PR — 8, and they are not cosmetic

All eight are tests/test_ndimage.py, which this PR does not modify. It modifies
src/_lcm/regime_building/ndimage.py, swapping

weighted_value = _multiply_all(weights) * contribution
# for
weighted_value = zero_safe_weighted_term(weight_product, contribution)

Verified directly rather than inferred from the logs — same file, same command, two trees:

tree	result
feat/continuous-outer @ db00f64	30 passed
collective-regimes @ 5c0b829	8 failed, 22 passed

There are two distinct causes.

(a) map_coordinates now returns a WRONG GRADIENT at every on-node coordinate — 1 test.

zero_safe_weighted_term(w, v) is w * where(w == 0, 0, v). Masking the value with a
hard select is sound when w is a constant — a Pareto weight, a regime-transition
probability, a quadrature weight. An interpolation corner weight is not constant: it
is a differentiable function of the very coordinate being differentiated. At an
exactly-integer coordinate one corner weight is exactly 0, where zeroes that corner's
value, and the term becomes w * 0 — whose derivative is w'(c) * 0 = 0 instead of
w'(c) * v. The gradient loses precisely the corner whose weight is changing.

Minimal witness, grid[i] = i² so the correct slope on [i, i+1] is grid[i+1] - grid[i]:

  c=1.0   value=1.000   grad=-1.000   on_node=True     (correct: 3.0)
  c=1.5   value=2.500   grad= 3.000   on_node=False    correct
  c=2.0   value=4.000   grad=-4.000   on_node=True     (correct: 5.0)
  c=2.5   value=6.500   grad= 5.000   on_node=False    correct
  c=3.0   value=9.000   grad=-9.000   on_node=True     (correct: 7.0)

  d/dw [zero_safe(w, 7.0)] at w=0.00 -> 0.0   (should be 7.0)  <-- the hard select
  d/dw [zero_safe(w, 7.0)] at w=0.25 -> 7.0
  d/dw [zero_safe(w, 7.0)] at w=1.00 -> 7.0

Off-node gradients are exact; on-node ones come back as -grid[c], i.e. only the
surviving corner's -1 * v_lo term. Values are unaffected — this is gradient-only, which
is exactly what makes it easy to miss. test_gradients[map_coordinates] catches it:
-10.0 where 2.0 is expected.

(b) The helper rejects integer weights — 7 tests.

BeartypeCallHintParamViolation: parameter weight="JitTracer(int32[7])"
violates type hint FloatND | ScalarFloat | float

The previous expression was a bare multiply with nothing to violate. All seven are the
int32 parametrization of test_map_coordinates_against_scipy /
test_map_coordinates_round_half_against_scipy; the float variants pass.

Suggested direction

(b) is an annotation/dtype-coverage fix. (a) is the substantive one and I do not think it
should be fixed by loosening the test: the zero-safe contract as written is a statement
about values, and map_coordinates needs it to hold for derivatives too. Either
map_coordinates keeps the plain multiply (its corner weights are never ±inf-valued,
so it may not need zero-safety at all), or zero_safe_weighted_term needs a
custom-JVP/stop_gradient-free formulation that preserves ∂/∂w at w == 0. Worth
checking whether any other caller passes a weight that is differentiable in the argument
of interest.

Inherited from the base — 68

Unchanged from db00f64; none of these are this PR's doing.

file	n
tests/test_solution_on_toy_model_stochastic.py	24
tests/test_solution_on_toy_model_deterministic.py	8
tests/simulation/test_subject_batching.py	7
tests/solution/test_n_nbegm.py	6
tests/solution/test_n_nbegm_finite_baseline.py	6
tests/test_processes.py	5
tests/solution/test_negm_bequest.py	3
tests/solution/test_negm_serviceflow.py	3
tests/solution/test_egm_carry.py	1
tests/solution/test_egm_discrete.py	1
tests/solution/test_n_nbegm_continuous.py	1
tests/solution/test_n_nbegm_epstein_zin.py	1
tests/solution/test_nbegm_distributed_carry_sharding.py	1
tests/test_regression_test.py	1

The seven test_subject_batching.py failures are worth calling out: they sit in
tests/simulation, which this PR modifies heavily, so they were my prior candidate for a
genuine regression. They fail identically on the base.

Method, and what is NOT covered

One process per chunk (peak RSS resets between them); -q -n 2 --tb=no -rf, with
### EXIT <chunk> = $? recorded after each so a chunk killed by the memory ceiling shows
up as unmeasured rather than silently counted as clean.

The four tests/test_mahler_yum_*.py files are UNMEASURED on both sides — that
chunk exited 143 (SIGTERM from the memory cap) on head and base, even at -n 1.
Symmetric, so it is unlikely to hide an asymmetry, but I have not measured it and am
not claiming it is clean.

The base was measured in a fresh detached worktree at db00f64, not in the shared
one: that worktree had uncommitted work in src/_lcm/egm/upper_envelope/query.py, the
same area as most of the inherited failures, so measuring there would have compared this
PR against someone's work in progress. _lcm.__file__ was checked to resolve inside each
tree, since a linked pixi env can otherwise resolve the package to a different branch.

[hmgaudecker]

Fixed — 27bc15f, and the failure set is now identical to the base

All 8 regressions from the diff above are closed. tests/test_ndimage.py is back to
30 passed, matching feat/continuous-outer @ db00f64.

The gradient one. zero_safe_weighted_term now masks only where the value is
non-finite, instead of on every zero-weight node. That keeps the property the helper
exists for while restoring ∂/∂w at w == 0:

• zero-mass safety unchanged — zs(0, ±inf) and zs(0, nan) are still exactly 0.0,
  and zs(1, -inf) is still -inf;
• d/dw [zs(w, 7.0)] == 7.0 at w = 0, 0.25 and 1.0 alike (it was 0.0 at w = 0);
• values are bit-identical to the previous form — for finite v, 0 * v == 0
  whether masked or not. Checked over 10,000 random cells including zero-weight slots
  with -inf on each of them, and then at suite scale: tests/solution returns the
  same 23 failures, node for node, as before the change. So this is gradient-only
  and carries no numerical risk to the EGM family.

The select still sits on an operand of the multiply, so the FMA-contractibility
argument in the module docstring is unaffected (docstring updated to say why the mask is
restricted).

The 7 annotation ones. Parameters and return were FloatND-only; map_coordinates
accepts FloatND | IntND, so an integer weight product raised
BeartypeCallHintParamViolation and then ...ReturnViolation. Widened.

New tests/regime_building/test_zero_safe_gradients.py pins all three legs — safety,
derivative, value-invariance — plus the map_coordinates-on-a-quadratic case that
started this.

Also in this push — 9f75cc6

Comparing every X / X_collective builder pair (there are exactly two in src/_lcm)
turned up a second instance of the class the branch-2 audit's F1 repaired:
get_Q_and_F_terminal_collective had no next_state_names parameter while its singleton
twin does and both call sites thread it. That argument feeds
_fail_if_unproduced_next_state_is_read, whose test is read_names & next_state_names,
so against the default frozenset() the guard could never fire on the collective path —
the model failed later with a bare missing-argument error instead of the named one.

Severity is diagnostic, not numerical, and I checked that rather than assuming it:
next_state_names reaches _get_U_and_F and is used only for the guard call, taking no
part in graph construction. Worth noting the corollary for F1 itself — of the four
arguments that repair restored, flow_transitions and continuation_functions are the
load-bearing ones and next_state_names was the diagnostic member; this is that member,
still dropped one function away.

Current state of this branch vs its base

	head 9f75cc6	base db00f64
failures	68	68
caused by this PR	0	—

Same caveat as before: the four tests/test_mahler_yum_*.py files remain unmeasured on
both sides (that chunk hits the memory ceiling even at -n 1), so this is "no
difference in what was measured", not "clean everywhere".

[hmgaudecker]

@/tmp/claude-1000/-home-hmg-econ-dev-pylcm-lcm-replications-src-lcm-reps-ecksteinCareerFamilyDecisions2019/da153776-cb2a-4e8a-9fc3-5b37c71b1f06/scratchpad/pr409-r3.md

[hmgaudecker]

Cascaded feat/continuous-outer in (f318eb0), local verification on the merged tree:

• tests/solution — 1250 passed, 3 skipped, 2 xfailed, 0 failed (run as four file chunks; see below)
• rest of suite — 1923 passed, 0 failed, 7 errors

The 38 DataFrame-shape failures this branch was carrying are gone. They came from d0d655c adding nested_policy_fallback without regenerating the pickled fixtures; 610b6f9 on #407 is the fix, and cascading it in resolved them. Nothing in the merge itself needed conflict resolution beyond the eight in regime_building/processing.py, resolved by provenance, plus a collective-twin signature widening in Q_and_F.py that ty caught and the suite did not.

The 7 remaining errors are #407's, not this branch's

tests/test_mahler_yum_continuous_outer.py and tests/test_mahler_yum_implicit_pilot.py error at setup with Out of memory allocating 158915772464 bytes — 148 GiB.

• Both files exist only on feat/continuous-outer and collective-regimes, not on main.
• Reproduced byte-identically (same allocation to the byte) on origin/feat/continuous-outer's own tip in a detached worktree, with _lcm.__file__ asserted to resolve into that worktree. So the merge did not introduce it.
• It is not contention or an -n artifact: a single test at -n 0 with no neighbours asks for the same 148 GiB, which exceeds the whole machine.

This would hit CI too. Both are @pytest.mark.slow, and CI phase 3 is pytest tests --ignore=tests/solution -m slow -n 2 — the same command shape, against a 16 GB runner. It is currently masked: Ubuntu dies at ~99% of phase 1 (-m "not slow") with The runner has received a shutdown signal — all dots, zero test failures — so the && chain never reaches phase 3, and macOS passes only because it deselects slow wholesale. Fixing the phase-1 memory problem would expose this one.

This looks like the class an in-flight external audit already has open (workspace proportional to n_rows * n_cells * max_runs causing supported-CI OOMs, escalation_required), so it is flagged here rather than re-diagnosed.

Note for anyone verifying locally

Do not try to phase tests/solution with -m "not slow" — it collects nothing, because the entire directory is slow. Chunk by file instead (ls tests/solution/*.py | split -n l/4, one pytest invocation per chunk); a single full-directory run gets OOM-killed at ~96% with no summary line.

[codecov]

Codecov Report

❌ Patch coverage is 88.25879% with 294 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.60%. Comparing base (b803c81) to head (723f9a2).

Files with missing lines	Patch %	Lines
benchmarks/asv/bench_collective_household.py	0.00%	173 Missing ⚠️
src/_lcm/user_regime_validation.py	89.69%	30 Missing ⚠️
src/_lcm/transition_checks.py	82.40%	19 Missing ⚠️
src/_lcm/pandas_utils.py	78.46%	14 Missing ⚠️
src/lcm/regime.py	96.11%	7 Missing ⚠️
src/_lcm/regime_building/canonicalize.py	81.81%	6 Missing ⚠️
src/_lcm/regime_building/V.py	94.04%	5 Missing ⚠️
src/_lcm/regime_building/phases.py	91.07%	5 Missing ⚠️
src/_lcm/solution/backward_induction.py	95.61%	5 Missing ⚠️
src/lcm/transition.py	89.13%	5 Missing ⚠️
... and 14 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #409      +/-   ##
==========================================
+ Coverage   92.05%   92.60%   +0.55%     
==========================================
  Files         310      331      +21     
  Lines       32809    37503    +4694     
==========================================
+ Hits        30201    34729    +4528     
- Misses       2608     2774     +166     

Flag	Coverage Δ
cpu-python	92.60% <88.25%> (+0.55%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[hmgaudecker]

Cascaded feat/continuous-outer in again (9b511cc2), and the branch is now
MERGEABLE against it after being CONFLICTING. The only conflict was the
_replay_continuous_outer_decision docstring in simulation/simulate.py: #407's
d191f850 replaced the single replay candidate with a scored candidate bank, so its
description is the accurate one, kept alongside this branch's sentence that inference
must refuse whenever any fallback entry is True.

Round 6 came back: eight of nine closed

F1-F8 are verified_fixed. F9 stays open, and it is worth being precise about what it
is: a certification finding about the certificate that establishes zero
represented-candidate search error, not a demonstrated wrong answer from the solver. No
production omission has been exhibited in any round, and both certified sources are
byte-identical between rounds 6 and 7 (git diff --name-only 4178176a b2d93944 -- src/
is empty).

Round 6 named two residual escapes. Both were reproduced here from the shipped
artifacts before anything was changed, rather than taken on the reviewer's word.

1. The compiled policy was blind to half the inventory. The
   profile-contract.policy.json shipped inside bundle-round6.zip contains 0
   occurrences of candidate_sources against 4 of candidate_source. The cause is
   structural, not a stale edit: the bundler compiles the contract down to one source
   and one source_digest per profile, so a second certified source declared under
   additional_sources cannot reach the policy the protocol enforces.

2. One fixed candidate ordering hid an omission class. The mask sweep was complete
   over all 63 nonempty feasibility neighbourhoods but ran at a single ascending
   ordering Q = [1,2,3,4,5,6]. Under a strictly ascending ordering the winner of any
   mask is its highest feasible index, so candidate 0 wins only the singleton mask and
   only candidate 5 is ever the full-support winner. An omission of any other candidate
   conditional on full support therefore changed nothing the certificate observed.

What changed

The contract's primary source/source_digest pair now names the generated inventory
and the SHA-256 of its bytes, with every certified source under additional_sources.
Because each source's digest is a record inside that inventory, changing either
source's bytes moves the inventory file, moves the anchor, and makes a stale policy
fail the join. That is the one shape a compiled policy can carry that still binds a set.

The certificate now sweeps the ordering as well as the mask: rank_vectors generates
one strict total order per candidate, each won by that candidate, and all four routes —
singleton and collective, solve and simulate — enumerate every ordering against every
nonempty mask against a scalar argmax that refuses ties and empty support. The AST gate
requires both matrices and both loops, so deleting the sweep is refused rather than
silently green.

Two controls ship with it, each requiring all five of its perturbations to be rejected:
mt7 seeds a byte change in each certified source, the anchor replaced by each, and an
absent anchor; mt8 collapses the ordering matrix and removes the sweep from each route
independently.

Evidence at b2d93944

Full suite on one 16-core node, pinned to the commit, fresh JAX compile cache per leg
(float64 0 -> 34877 entries, float32 0 -> 33209), SLURM COMPLETED in 00:33:22
with exit code 0:0:

precision	tests	passed	failures	errors	skipped	time
64	7090	7028	0	0	62	1193.6 s
32	7055	6732	0	0	323	779.8 s

Test and skip counts are identical to round 6 at both precisions, which is expected:
this round changed the content of four tests, not their number. The candidate
certificate is 50 passed / 0 failed at both precisions, and prek --all-files is clean
(34 hooks, 0 failed).

One thing that moved the wrong way

The protocol's own anchor checker now reports all_structural_markers_present: false
where round 6 reported true. This is a direct consequence of the anchor change: that
tool greps the file named by candidate_source for full-grid enumeration markers, and
that file is now a JSON inventory. Nothing about the production sources changed. The
enumeration route is instead resolved through the syntax trees of both certified
sources by the certificate's structural half, which is strictly more than a marker grep
over one file — but the tool prints false, and that is recorded rather than
paraphrased away.

The earlier OOM report in this thread is from 2026-07-29 and describes a tree many
commits behind; treat this comment as superseding it.

[hmgaudecker]

@timmens Final close-out: the current head is 723f9a26 and the external correctness audit is now definitively CLEAN / CLOSED after nine rounds. All nine findings are verified_fixed; there are no open findings, blockers, or pending decisions.

The last residual item was certificate architecture, not a demonstrated production error. The final form proves exact Q_arr/F_arr transport through all six ordinary/taste-shock singleton/collective solve/simulation corridors, seals the 43-source helper/reducer frontier, and rejects 66/66 semantic plus 176/176 synchronized full mutations (including the historical MT9/MT10 escapes).

Round-9 verification at 1bff3e16: candidate certificate 93/93 at each precision; full float32 6,787 passed / 316 skipped / 7 xfailed; full float64 7,083 passed / 55 skipped / 7 xfailed; zero failures/errors; all 34 pre-commit hooks passed.

The follow-up head 3272af34 fixes a Windows-only certificate portability defect: sealed UTF-8 source text is now hashed after newline canonicalization, so a CRLF checkout no longer produces false mismatches while non-newline mutations remain detected. Its expanded certificate battery passes 95/95 at both precisions; inventory generation, the unified contract/policy verifier, all 176 synchronized mutations, and a real core.autocrlf=true clone pass as well. The follow-up changes only three test/tooling paths and no production src/ path.

This supersedes my Round-7 status comment above.

The follow-up to review 5064008752 was implemented at 90fe284f and is included in this head. It preserves ordinary DAG
ancestors in carried-state Pareto imputation, retains narrow reachability for Phased
per-target transitions, and replaces the two public Model docstring references to the
lowered edge type with ValueDependentTransition. It also incorporates #407 at
1c9f1d22 and re-anchors the certificate to the combined tree. After #407 landed on
main at b803c810, the tree-preserving post-land cascade produced 723f9a26; the
inventory and unified verifier remain green. PR #409 was then squash-merged into
main at e7e4a83f. Focused combined-tree
verification is green: float32 193 passed / 5 skipped; float64 198 passed; zero failures
or errors in either profile. The unified verifier passes and all 176/176 synchronized
direct-flow mutations are rejected.

[timmens]

Thanks for the substantial follow-up. I reviewed the current head 1bff3e16b from
494a3a5c7, reran the previous-finding battery, and checked the declaration redesign
and candidate certificate. The earlier joint-transition, callable-probability,
zero-argument-weight, and duplicate-declaration-surface concerns are resolved.

I am approving conditionally on the required CI becoming green. Windows currently
fails because the candidate certificate hashes raw checkout bytes, so CRLF line endings
change the source seals; the benchmark job is still running. The remaining review
points below are non-blocking, and I am happy for you to decide whether they belong in
this PR or a follow-up.

Non-blocking correctness follow-ups

Preserve ordinary DAG dependencies of carried-state Pareto weights

A Pareto weight may now read a carried state, which fixes the direct case. The carried
state's solve imputation is also a first-class regime function, so it may read another
function in the regime DAG. In a small public model I used weight_f(power),
impute_power(power_signal), and a normal regime function named power_signal. Model
construction succeeds, then solve(log_level="debug") raises
KeyError: 'power__power_signal'.

_compose_carried_imputations includes the carried imputations alone and consequently
reclassifies the ordinary function read as a parameter. Could the weight evaluator
compose the reachable solve-function DAG while preserving the existing
function/parameter distinction? A regression test can assert the resulting stakeholder
values (3, 0). This is a specialized, immediately visible failure with a simple user
workaround, so I do not consider it a merge blocker.

Preserve narrow reachability for Phased target mappings

The folded-conditioner guard appears to lose the declared target set when the regime
transition is Phased. I reproduced this with two disconnected paths:
safe_entry -> folding holds risk_type fixed, while
moving_entry -> sideways changes it. moving_entry.transition is
Phased(solve={"sideways": ...}, simulate={"sideways": ...}). Model construction
rejects the valid folding path and says moving_entry reaches folding.

_reachable_regime_targets recognizes a direct mapping and treats the outer Phased
object as a coarse transition over every regime. Could it resolve either phase first?
The phase grammar already guarantees matching forms and target keys. This requires a
rare combination of features and fails during model construction, so I also consider
it non-blocking.

Small documentation cleanup

The single declaration surface looks much clearer to me. One small remnant remains:
the public Model.solve and Model.simulate docstrings describe dissolution in terms
of GatedEdge, while that class is now internal and users declare a
ValueDependentTransition. Could those two references use the public name and
vocabulary?

Focused local checks at this head: 64 previous-finding tests passed, 405 declaration
and integration tests passed, and all 93 candidate-certificate tests passed.