Skip to content

Typeset by format, and say what the render leaves out - #951

Open
FBumann wants to merge 3 commits into
spec-builderfrom
spec-typeset-api
Open

Typeset by format, and say what the render leaves out#951
FBumann wants to merge 3 commits into
spec-builderfrom
spec-typeset-api

Conversation

@FBumann

@FBumann FBumann commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Note

The following content was generated by AI.

Stacked on #950. The typesetting API, in three commits.

1. The format is an argument, not six methods

math-spec spells its typesetter as typeset(model, fmt) over a FORMATS
registry, and typeset_declaration(model, name, fmt) the same way. linopy
re-expanded that one axis into six methods — three on ModelSpec, three on
Declaration — each a one-line delegation. linopy's public surface therefore
tracked math-spec's list of formats: a fourth format upstream meant either two
more methods here or a silent asymmetry. **options was passed through
untyped too, so standalone=, legend=, numbered=, symbols= were
invisible at the linopy level.

typeset(fmt) is the method now, on both classes, with to_latex,
to_markdown and to_typst kept as three-line aliases — they read the way
pandas taught everyone to expect, and a notebook repr needs one. A format
math-spec adds is reachable the day it lands, with no release here.

The docstrings also said "the whole model", which the code never delivered —
it renders the spec. They say the spec now.

2. Say what the render leaves out

A spec-built model need not be only its spec: add_variables and
add_constraints go on working on one, and what they add carries no math-spec
declaration to typeset. Nothing said so — a hybrid model rendered to LaTeX
silently dropped every hand-added variable and constraint. That is the kind of
omission that survives review and reaches a paper.

>>> m = Model.from_spec(spec, sources)
>>> m.add_variables(coords=[hand], name="hand_made")
>>> m.add_constraints(m.variables["hand_made"] <= 1, name="hand_limit")
>>> m.spec.unspecified
Unspecified(variables=('hand_made',), constraints=('hand_limit',))
>>> print(m.spec.to_latex())
UserWarning: this model holds 1 variable (hand_made) and 1 constraint (hand_limit)
added outside the spec, which carry no math-spec declaration and are not typeset,
so this is not the whole model.
% Added outside this spec and not shown: 1 variable (hand_made) and 1 constraint (hand_limit).
\paragraph{Sets}
...

model.spec.unspecified reports it in one place — Model.__repr__ computed the
same difference inline to decide its [spec] tags and now reads it too. Where it
is non-empty, typesetting says so three ways, because no one channel reaches
every reader:

  • a UserWarning, for the script that renders a file;
  • a comment of the format's own opening the rendered text — % in LaTeX,
    <!-- in Markdown, // in Typst — gone once compiled, there for whoever reads
    the source;
  • a visible line under the Markdown a notebook displays, which is the one
    place a warning is routinely swallowed.

A format math-spec adds that we have no comment syntax for renders without one
rather than with a wrong one; the warning still fires. Declaration is left
alone — it is reached by name through the spec, so it can never be out of step
with the model.

What is not reported

A spec declaration the data emptied — a variable under a where: no row
satisfies — still typesets, with its where: clause intact. That is correct: the
spec is the mathematics and the data is the data, and the condition is right
there in the output. Only the model-has-more direction is drift.

3. Everything a model can hold, not just variables and constraints

The first pass compared names, and two of the things a model can hold have no
name to compare:

  • SOS. add_sos_constraints writes sos_type / sos_dim onto a variable
    that is already there — no new name, so name-comparison could never see it.
  • The objective. add_objective mutates the objective in place. A model
    whose objective was replaced after the build reported no drift at all, and the
    render then shows the spec's objective, which is no longer the model's — wrong
    rather than incomplete.

Two more were missed for duller reasons:

  • Named expressions. A spec's own are read lazily off model.spec and never
    live in model.expressions, so everything that does live there was added by
    hand and none of it was reported.

  • Piecewise, at the wrong altitude. add_piecewise_formulation puts four
    generated constraints in the model and the tally named all four rather than the
    formulation. Model.__repr__ already groups them via _get_piecewise_groups;
    unspecified now reads the same grouping, so variables and constraints are
    left to what a caller actually declared:

    before:  constraints=('curve_delta_bound', 'curve_fill_order',
                          'curve_binary_order', 'curve_link')
    after:   constraints=(), piecewise=('curve',)
    
@dataclass(frozen=True)
class Unspecified:
    variables:   tuple[str, ...]
    constraints: tuple[str, ...]
    expressions: tuple[str, ...]
    sos:         tuple[str, ...]   # variables given an SOS the spec did not declare
    piecewise:   tuple[str, ...]   # formulation names, not their internals
    objective:   bool

The objective is a flag rather than a comparison, since the object is mutated in
place: add_objective sets it when the model already carries a spec, which a
build cannot trigger because it runs before _spec is assigned. A copy carries
it, and a netcdf file records it as an attribute — written only when true, so a
file from an untouched spec is byte-for-byte what it was. Nothing else in the
file could say the typeset objective is not the model's.

No false drift on a spec's own constructs

Verified, because this is the way the design could have failed: math-spec lowers
piecewise: and sos: into ordinary declarations, so they sit in the program
like anything else and typeset like anything else.

program.variables   : ['p', 'op_cost', 'cost_curve_lam']
program.constraints : ['balance', 'cost_curve_chord', 'cost_curve_domain_lo', ...]
program.sos         : {'cost_curve': SosDeclaration(variable='cost_curve_lam', sos_type=2)}
unspecified         : falsy
Verification (f08408c)
uv run --extra dev --extra solvers --group spec pytest test/ -q --ignore=test/remote
9068 passed, 1074 skipped

uv run --extra dev --group spec mypy linopy
Found 13 errors in 3 files (checked 49 source files)
# identical to the base commit: missing types-PyYAML and pre-existing
# linopy/expressions.py union-attr errors in this local environment

test/remote is excluded: its OETC cloud-provider tests fail identically on the
base commit in this environment and touch nothing here.

Checklist

  • AI-generated content is marked (see AGENTS.md).
  • Code changes are sufficiently documented.
  • Unit tests for new features were added.
  • A note for the release notes doc/release_notes.rst is included.

🤖 Generated with Claude Code

https://claude.ai/code/session_015ZfHfTFENUy6WxnFFri5Td

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown

Build cost — v1 vs legacy

v1 build peak & time relative to legacy, on this commit — not a comparison against master (that is CodSpeed).

peak — v1 / legacy time — v1 / legacy
peak v1/legacy time v1/legacy
Full table (time + peak, mean)
benchmarks/drivers/test_build.py::test_build[basic-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.08651 (1.07)   │   15.03 (1.00) 
 (v1)        0.08112 (1.0)   │    15.00 (1.0) 

benchmarks/drivers/test_build.py::test_build[basic-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.09257 (1.06)   │   12.04 (1.00) 
 (v1)        0.08715 (1.0)   │    12.04 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=0]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03786 (1.09)   │   15.20 (1.0) 
 (v1)        0.03462 (1.0)   │   15.20 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=100]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.05457 (1.13)   │   44.93 (1.0) 
 (v1)         0.0485 (1.0)   │   44.93 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.04084 (1.08)   │   11.51 (1.0) 
 (v1)        0.03788 (1.0)   │   11.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.09718 (1.04)   │   24.34 (1.06) 
 (v1)        0.09322 (1.0)   │    23.04 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=250]
                 time (s)         peak (MiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.1072 (1.05)   │   16.12 (1.00) 
 (v1)        0.1021 (1.0)   │    16.12 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=10000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.02311 (1.05)   │   752.18 (1.10) 
 (v1)        0.02202 (1.0)   │    685.15 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=100]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.02276 (1.06)   │   3.12 (1.33) 
 (v1)        0.02151 (1.0)   │    2.34 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=0]
                 time (s)          peak (MiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.0642 (1.23)   │   126.16 (1.44) 
 (v1)       0.05239 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=100]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.06328 (1.24)   │   126.16 (1.44) 
 (v1)        0.05101 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=50]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.06382 (1.24)   │   126.16 (1.44) 
 (v1)         0.0515 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[masked-n=100]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)    0.056 (1.06)   │    715.12 (1.0) 
 (v1)       0.05292 (1.0)   │   787.73 (1.10) 

benchmarks/drivers/test_build.py::test_build[masked-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.05243 (1.09)   │   4.54 (1.27) 
 (v1)         0.0482 (1.0)   │    3.57 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=0]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.3663 (1.05)   │   704.12 (1.09) 
 (v1)        0.3498 (1.0)   │    643.85 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=100]
                 time (s)        peak (MiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.3838 (1.04)   │   18.34 (1.0) 
 (v1)        0.3676 (1.0)   │   18.34 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=50]
                 time (s)       peak (MiB) 
 name                mean   │         mean 
───────────────────────────────────────────
 (legacy)   0.3813 (1.04)   │   9.54 (1.0) 
 (v1)        0.3658 (1.0)   │   9.54 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.07385 (1.08)   │   3.77 (1.12) 
 (v1)        0.06813 (1.0)   │    3.37 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=50]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.07389 (1.08)   │   216.59 (1.10) 
 (v1)         0.0684 (1.0)   │    196.23 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=0]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.03737 (1.08)   │   938.49 (1.0) 
 (v1)         0.0345 (1.0)   │   938.49 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=100]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.03865 (1.09)   │   9.66 (1.0) 
 (v1)         0.0354 (1.0)   │   9.66 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=50]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.03808 (1.09)   │   5.32 (1.0) 
 (v1)        0.03495 (1.0)   │   5.32 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance_sparse-severity=0]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.02067 (1.01)   │   1.47 (1.0) 
 (v1)        0.02049 (1.0)   │   1.47 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance_sparse-severity=100]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.02057 (1.00)   │   1.47 (1.0) 
 (v1)        0.02048 (1.0)   │   1.47 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance_sparse-severity=50]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.02058 (1.00)   │   1.47 (1.0) 
 (v1)        0.02058 (1.0)   │   1.47 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=1000]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.1828 (1.04)   │   946.85 (1.06) 
 (v1)        0.1762 (1.0)   │    891.54 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=10]
                 time (s)         peak (KiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.1803 (1.05)   │   12.01 (1.00) 
 (v1)        0.1722 (1.0)   │    11.99 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.04629 (1.06)   │   147.70 (1.06) 
 (v1)        0.04364 (1.0)   │    139.87 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.04593 (1.06)   │   2.60 (1.09) 
 (v1)        0.04325 (1.0)   │    2.38 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=0]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03825 (1.08)   │   696.75 (1.03) 
 (v1)         0.0355 (1.0)   │    673.70 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=100]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)    0.08395 (1.0)   │   137.97 (1.0) 
 (v1)       0.08486 (1.01)   │   137.97 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.06014 (1.07)   │   69.22 (1.0) 
 (v1)        0.05619 (1.0)   │   69.22 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.04449 (1.10)   │   402.33 (1.00) 
 (v1)        0.04045 (1.0)   │    402.30 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.04402 (1.10)   │   3.19 (1.19) 
 (v1)        0.03994 (1.0)   │    2.69 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.04817 (1.06)   │   29.00 (1.54) 
 (v1)        0.04527 (1.0)   │    18.84 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.05857 (1.11)   │   37.95 (1.43) 
 (v1)        0.05267 (1.0)   │    26.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[storage-n=10]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)    0.09252 (1.0)   │    410.93 (1.0) 
 (v1)       0.09368 (1.01)   │   427.84 (1.04) 

benchmarks/drivers/test_build.py::test_build[storage-n=250]
                 time (s)         peak (MiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.09788 (1.0)   │     9.94 (1.0) 
 (v1)       0.1007 (1.03)   │   10.22 (1.03) 

📊 Interactive plots + CSV: download the semantics-report-v1-vs-legacy artifact from this run.

Report-only · not a gate · refreshed on every push · obsolete once legacy is dropped.

Base automatically changed from spec-review-fixes to spec-builder September 8, 2026 17:25
FBumann and others added 3 commits September 8, 2026 19:25
math-spec spells its typesetter as `typeset(model, fmt)` over a `FORMATS`
registry, and `typeset_declaration(model, name, fmt)` the same way. linopy
re-expanded that one axis into six methods, three per class, each a one-line
delegation -- so linopy's public surface tracked math-spec's list of formats,
and a fourth format upstream would have meant either two more methods here or
a silent asymmetry.

`typeset(fmt)` is the method now, on `ModelSpec` and on `Declaration` alike,
with `to_latex`, `to_markdown` and `to_typst` kept as the three-line aliases
that spell today's formats -- they read the way pandas taught everyone to
expect, and a notebook repr needs one. A format math-spec adds is reachable
the day it lands, without a release here.

The docstrings said "the whole model", which the code never delivered: it
renders the spec, and a spec-built model can hold more than its spec. They
say the spec now; making the difference visible is the next commit's job.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZfHfTFENUy6WxnFFri5Td
Typesetting renders the spec, and a spec-built model need not be only its
spec: `add_variables` and `add_constraints` go on working on one, and what
they add carries no math-spec declaration to typeset. Nothing said so. A
hybrid model rendered to LaTeX silently dropped every hand-added variable and
constraint -- the kind of omission that survives review and reaches a paper.

`model.spec.unspecified` names them, in one place: `Model.__repr__` computed
the same difference inline to decide its `[spec]` tags and now reads it too.
Where it is non-empty, typesetting says so three ways, because no one channel
reaches every reader:

- a `UserWarning`, for the script that renders a file;
- a comment of the format's own opening the rendered text -- `%` in LaTeX,
  `<!--` in Markdown, `//` in Typst -- gone once compiled, there for whoever
  reads the source;
- a visible line under the Markdown a notebook displays, which is the one
  place a warning is routinely swallowed.

A format math-spec adds that we have no comment syntax for renders without
one rather than with a wrong one; the warning still fires. `Declaration` is
left alone: it is reached by name through the spec, so it can never be out of
step with the model.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZfHfTFENUy6WxnFFri5Td
`unspecified` compared names, and two of the things a model can hold have no
name to compare. `add_sos_constraints` writes `sos_type` and `sos_dim` onto a
variable that is already there, and `add_objective` mutates the objective in
place -- so a hand-added special-ordered set, and an objective replaced after
the build, both changed the mathematics while `unspecified` stayed empty. The
objective is the worse of the two: the render then shows the spec's objective,
which is no longer the model's, so it is wrong rather than incomplete.

Named expressions were missed for a duller reason: a spec's own are read
lazily off `model.spec` and never live in `model.expressions`, so everything
that does live there was added by hand and none of it was reported.

Piecewise was reported, at the wrong altitude -- `add_piecewise_formulation`
puts four generated constraints in the model, and the tally named all four
instead of the formulation. `Model.__repr__` already groups them through
`_get_piecewise_groups`; `unspecified` now reads the same grouping and names
formulations, leaving `variables` and `constraints` to what a caller actually
declared.

The objective is a flag rather than a comparison, since the object is mutated
in place: `add_objective` sets it when the model already carries a spec, which
a build cannot trigger because it runs before `_spec` is assigned. A copy
carries it, and a netcdf file records it as an attribute -- written only when
true, so a file from an untouched spec is byte-for-byte what it was. Nothing
else in the file could say the typeset objective is not the model's.

None of this fires on a spec's own constructs: math-spec lowers `piecewise:`
and `sos:` into ordinary declarations, so they sit in the program like
anything else and typeset like anything else.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZfHfTFENUy6WxnFFri5Td
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant