DOC Reorganize documentation around modeling decisions - #427
Conversation
…ve-regimes # Conflicts: # src/_lcm/egm/upper_envelope/query.py # src/_lcm/optimization/implicit_outer_derivative.py # src/_lcm/simulation/result_dataframe.py # tests/solution/test_envelope_query.py # tests/test_continuous_outer_audit_regressions.py
…nto feat/perceived-stochastic-transitions
…tions' into feat/continuous-outer
…nto feat/perceived-stochastic-transitions
…tions' into feat/continuous-outer # Conflicts: # .github/workflows/cpu.yml
…ective-regimes # Conflicts: # src/_lcm/solution/nnbegm.py # tests/solution/test_nbegm_flat_continuation.py
…ective-regimes # Conflicts: # tests/ci/test_cpu_workflow_contract.py
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
Documentation build overview
72 files changed ·
|
#409's base had moved five commits ahead, leaving the pull request `CONFLICTING` / `DIRTY`. GitHub could not build a merge ref for it, so pre-commit.ci reported "error during mergeable check" and no pull-request workflow ran at all — the branch looked untested rather than failing. The only content conflict was the `_replay_continuous_outer_decision` docstring, where both sides had rewritten the return description. The signature returns four values, so the four-item list is the accurate one; the sentence about inference refusing on a fallback entry is kept from the other side rather than dropped. `feat/continuous-outer` also installs the hooks that live in the `.ai-instructions` submodule and adds codespell. Three sites here needed attention: the two deliberate `probabilit` regex prefixes take the inline `codespell:ignore` this repository already uses for that shape, and one docstring now spells "redeclares". That last one moves a certified source, so the candidate-certificate inventory and the profile contract were regenerated from it — which is the architecture working: a one-word docstring edit turned both `generate_sources.py --check` and the unified verifier red before either was told anything had changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gzx6oLKVkrBrxLXhseoznK
… bank When a refined nested proposal cannot be replayed at a subject's realized state, simulation falls back to the branches the solve published: the keeper plus one conditional inner policy per outer mesh node. It now emits the canonical-Q maximizer over the branches that are both reachable and canonically feasible there, instead of the branch carrying the highest value the solve happened to publish for it. Three things change together, because each alone leaves a way for a valid published branch to be discarded. The selector returns the complete bank rather than one winner. Its rows are node-major, keeper first, each carrying its own conditional inner action, and its mask covers only replay structure, interpolation support, finiteness and the solver-owned consumption budget. `_nested_grid_baseline` then scores every surviving row through canonical Q in one vmapped call and drops infeasible or non-finite rows before the argmax. The winner used to be chosen first and scored afterwards, so a branch the realized state makes infeasible could take the selection and then be rejected, discarding the rest of the bank with it. Ranking on published values could also prefer a branch that canonical Q ranks lower. Both the bank predicate and the per-pair admissibility check now read one domain, the published outer mesh. They used the declared inverse domain and the mesh extrema respectively, so a keeper landing in the gap between a wide declared domain and a narrower mesh was admitted by the first and rejected by the second, suppressing a reachable node that nothing reconsidered. Tie order is explicit: the keeper is row zero and wins an exact tie, then ascending mesh-node order; the grid baseline wins an exact tie against the best published row, preserving the no-degradation convention. When no branch survives the caller raises before emitting an action. The accompanying test module covers a lower-published-value feasible branch, a published/canonical order reversal, both tie conventions, inner-policy alignment, an all-infeasible bank, and a mesh narrower than the declared domain, each at both precisions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Akf3HYBxuhGjKftH8jofDq
…rdering A compiled profile policy carries one path and one digest per profile, so a certified set declared under `additional_sources` never reached the policy the protocol actually enforces. The contract's primary `source`/`source_digest` pair now names the generated inventory and its bytes, which every certified source feeds, so the compiled policy binds the whole set: change either source and the inventory moves, and the stale policy is rejected. `verify.py --policy` reads both the rich and the compiled projection and joins the anchor exactly. The executable half swept only the feasibility mask, at one fixed ascending ordering. Under that ordering the winner of a mask is its highest feasible index, so candidate 0 wins no multi-feasible mask and only the last candidate is ever the all-feasible winner — an omission of any other candidate changes nothing the certificate observes. `rank_vectors` generates one strict total order per candidate, each won by that candidate, and all four routes now enumerate every ordering against every nonempty mask. The AST gate requires both matrices and both loops, so removing the sweep is refused rather than silent. mt7 seeds a byte change in each certified source, an anchor replaced by each, and an absent anchor; mt8 collapses the ordering matrix and removes the sweep from each route, and exhibits the miss the fixed ordering admits. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gzx6oLKVkrBrxLXhseoznK
Selection now ranks the keeper and every published node by canonical Q, so the branch values the fixture published no longer decide which branch a fallback emits. The module steered selection with those values, which left ten tests asserting the superseded rule and three more passing whichever rule ran: the stub objective `-|investment|` makes the keeper the unconditional argmax. Each test now names the outer action its objective favours, and a refusal is made observable by peaking the objective at the branch the rule must refuse -- without that the favoured branch loses on value anyway and the test reports the same outcome whether the rule excluded it or not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Akf3HYBxuhGjKftH8jofDq
…bank Simulation no longer keeps the grid-argmax pair when an off-grid nested read is refused. The grid pair and every published replay branch are scored by the canonical Q, the higher admissible score is emitted, and the grid pair takes an exact tie. The DataFrame column's description said the grid pair was kept, which was true before the adaptive replay fallback and is not true now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gzx6oLKVkrBrxLXhseoznK
…cs-information-architecture
|
@timmens Final verification after the last stack update: the implementation reply at The post-land bridge passes At the documentation verification head |
Windows checkouts use CRLF, so raw-byte digests rejected semantically identical certified sources. Hash canonical UTF-8 text while retaining sensitivity to non-newline changes.
…ocs-information-architecture
…ocs-information-architecture
timmens
left a comment
timmens
left a comment
There was a problem hiding this comment.
Approved. All follow-up comments below are non-blocking.
Medium
Finite outer search describes the wrong quantity as grid-snapped
docs/reference/outer_search.md:18
FiniteOuterGrid.grid contains outer post-decision targets. Simulation recovers the
declared outer action by exactly inverting the post-decision map. For the supported map
target = old + 2 * action, selecting target 1 when old = 0 yields action 0.5,
so the recovered action can lie between action-grid nodes. Could this say that the
selected post-decision target is grid-snapped and that the action is recovered exactly?
Describe refined_grid_factor as envelope-row headroom
docs/reference/solvers.md:107
refined_grid_factor sizes the NaN-padded envelope rows. It provides storage headroom
for ownership changes, and a row that needs more slots is reported as overflow and
NaN-poisoned. The density of the policy read-out grid stays unchanged. Could this
describe the field as envelope-row headroom so users tune the correct numerical
resource?
Qualify the introductory Bellman equation
docs/methods/dynamic_programming.md:7
The displayed u + beta E[V] recursion describes the default LinearAggregator() plus
LinearExpectation() specification. pylcm also supports the general
H(u, CE(V')) recursion through, for example, CESAggregator() and PowerMean().
Could this either qualify the equation as the default linear case or display the general
aggregator/certainty-equivalent form used on the preferences page?
Clarify joint-probability validation and normalization
docs/reference/transitions.md:103
The statement that joint-transition probabilities are never silently normalized does
not hold when validation is disabled. With probabilities [0.2, 0.3],
solve(log_level="off") completes and the default certainty equivalent returns the
mass-normalized mean. In a two-node witness with values [0, 3], the result is 1.8:
(0.2 * 0 + 0.3 * 3) / (0.2 + 0.3). Could this explain that debug rejects invalid
mass, warning and progress warn and continue, and off skips the check? Any path
that continues into aggregation normalizes the mass it receives.
State the deterministic factory contract for age specialization
docs/user_guide/age_specialized.md:39
The list of user obligations is missing that build(age) must be deterministic and
side-effect-free. Model construction resolves the same age multiple times. A minimal
counter-based factory was called four times for age 20, and the installed utility
captured the fourth result. A stateful factory can therefore validate one object and
install another. Could this third correctness obligation be stated here and in the
linked Reference section?
Document all persistence artifacts and the consuming save contract
docs/user_guide/solving_and_simulating.md:350
SimulationResult.save() writes four sibling artifacts. arrays/ contains the
per-subject raw-result tree, while the solution arrays are stored separately in
V_arr/; the current three-item list omits V_arr/ and assigns its sharded-value
behavior to arrays/. save() also consumes the in-memory result by clearing its value
function arrays and compiled regimes, so callers needing further access must reload.
Could this section match the four artifacts and state that consuming behavior?
Validate the rendered target of public API links
tests/test_docs_public_api_coverage.py:51
This check strips each fragment and verifies only that the destination file exists, so
it misses MyST resolving a duplicate implicit heading ID to a different page. In the
current built site, lcm.Model resolves to Tiny Example, the continuous-grid links to
the User Guide grids page, the state-transition links to the User Guide transitions
page, and NEGM/NNBEGM to the Nested EGM methods page. Could the canonical pages use
unique explicit labels and could this guard assert the rendered target source as well
as file existence?
Low
Show the keyword-only standalone persistence calls
docs/reference/runtime_and_results.md:67
Both standalone persistence functions are keyword-only, so the displayed
save_solution(solution, path) and load_solution(path) spellings raise TypeError
when copied. Could this show the callable forms
save_solution(period_to_regime_to_V_arr=..., path=...) and
load_solution(path=...)?
Include qualified constructor calls in the documentation sweep
tests/test_docs_constructor_kwargs.py:47
The AST walk accepts only ast.Name, so it checks Model(bogus=...) while silently
skipping lcm.Model(bogus=...). Several prominent reference examples use qualified
constructors, including lcm.Model, lcm.AgeGrid, lcm.LiquidMargin, and
lcm.OuterContinuousMargin, which leaves their keyword arguments outside this guard.
Could the collector resolve supported qualified names so the claimed documentation-wide
contract covers those examples?
Remove control characters from the joint-transition docstring
src/lcm/regime.py:173
There are two embedded DC4 (U+0014) control characters around “including carried-only
simulation states.” These can render unpredictably in generated API documentation and
source viewers. Could they be replaced with ordinary punctuation?
2 participants
What problem do you want to solve?
The existing documentation mixed task-oriented guidance, mathematical explanation,
examples, exact API contracts, and development internals. That made the User Guide too
deep for new users and left the EGM solver family without a coherent authoring journey.
This PR reorganizes the book into six distinct chapters:
It makes EGM, DCEGM, NEGM, NBEGM, and NNBEGM first-class solver families; adds a
problem-first specialized-solver authoring path; separates solver prerequisites such as
conditions, case pieces, piecewise-affine schedules, and margin declarations into
cross-linked Reference pages; and adds curated specialized consumption-saving and
collective-regime examples.
The change also:
controls that do not exist;
The documentation and supporting contracts went through five bounded external review
rounds. The final review is closure-clean with no remaining Serious or Moderate finding.
Todo
collective-regimes).723f9a26; review repairs at90fe284f, Inference-grade continuous outer choice (GSS) for NNBEGM #407 landed onmainatb803c810).mainate7e4a83fand cascade that tip into this branch at6fb4c87e.routing in float64 and float32.
Verification
Documentation verification at
810be995, after merging the Round-9 #409 closure head1bff3e16. Commit85330260then merged the #409 three-file Windows certificatenewline-portability follow-up. Commit
6dc03386merged the #409 repair head90fe284f, including review5064008752 repairs and #407 at
1c9f1d22. After #407 landed onmainatb803c810, commit771e174fcascades the tree-preserving #409 post-land head723f9a26. PR #409 then landed onmainate7e4a83f; commit6fb4c87ebridges that squash back into this branch. The bridge is tree-identical tothe verified pre-land head, and its per-step checks pass:
prek run ty --all-files: passed.pixi install -e tests-cuda12 --locked: passed with no lockfile change.prek run --all-files: exit 0, 34 hooks, every onePassed, no file modified.pixi run -e docs build-docs: exit 0, 70 pages, no MyST or cross-reference errors..md/.ipynbfiles(excluding
_build), every name imported fromlcmin a fenced Python block ornotebook code cell resolves against
dir(lcm). This checksfrom lcm import ...specifically; it does not cover attribute access after a bare
import lcm, submoduleimports, or API named only in prose.
GatedEdge,stakeholders=,pareto_objective=,value_constraints=,same_period_refs=,gated_edges=) no longer appears indocs/or
src/lcm_examples/. Two references survived the merge and were removed in afollow-up commit: the public-API index still listed
lcm.GatedEdge, and thecollective-regimes reference still described declaring an edge in either spelling. The
index entry was a genuine failure of
tests/test_docs_public_api_coverage.py::test_public_api_index_covers_exactly_the_lcm_exports,which compares the documented set against
lcm.__all__in both directions; that testis green again (5 passed, 0 failed).
prekdoes not cover it, which is why the mergepassed the hook gate with the test red.
this branch then adds documentation-only changes, whose affected documentation, example, API,
and routing seams were exercised directly.
passed / 5 skipped; float64 198 passed; zero failures or errors in either profile.
The unified verifier passes and the synchronized self-test rejects all 176/176
direct-flow mutations.
docs/andsrc/lcm_examples/found no stale replay semantics orpublic use of the lowered edge vocabulary. The remaining
GatedEdgemention explainsthat it is engine-internal, so no documentation change is required for this merge.
Merge resolution worth noting
Merging #409 into this branch conflicted in
docs/user_guide/defining_models.md, wherethis branch had replaced the inline
JointTransitionblock with a pointer toreference/transitions.md#joint-transitionswhile #409 had grown that block.The resolution keeps this branch's pointer, but the pointer's target did not yet cover
four things #409 had added, so they were moved into the reference page rather than
dropped: what
support,probabilitiesand output laws may read, together with theparams-bound preflight and the guarantee that probability vectors are never silently
normalized; the
params[source][target][kernel][...]parameter paths; and outputs ontoa stochastic process, including the node-basis reading and the
NaNa value outside thesupport yields.