Skip to content

feat(release): APR-RELEASE-001 §5 P0·Instrument — make build-report reads the 1092 ledger records nothing read - #3271

Open
noahgift wants to merge 6 commits into
mainfrom
PMAT-1108-build-report
Open

noahgift wants to merge 6 commits into
mainfrom
PMAT-1108-build-report

Conversation

@noahgift

Copy link
Copy Markdown
Contributor

What

APR-RELEASE-001 §5 P0·Instrument's Done test is "≥ 20 records; make build-report runs on a clean checkout". Measured on main: 1092 ledger records are committed under docs/build-ledger/ in exactly the P0·Instrument schema — 1084 of them job records — and nothing in the tree read one. So the first half passes 54× over and the second half had no mechanism, which is why §1's p95, §1's max-PRs-per-train and the whole of §7's gate: line were still [U] while the data sat in git.

This is the sccache class: recorded thousands of times, never compared.

scripts/build_report.sh reads them in 0.27 s, byte-identical across runs.

What it says — the numbers that were [U] until now

host_class n p50 total_s p95 total_s p50 queue_s p95 queue_s
gx10 509 366 2282 137 1399
intel 454 666 4994 34 1066
yoga 121 340 3518 11 832
gate:    p95 ci/gate 20.1 | max PRs/train 120 | queue p95 intel 1066 yoga 832 gx10 1399

Two things fall out of that table that no occupancy average could show. gx10 carries more jobs than intel (509 vs 454) and has the worst queue p95 (1399 s vs 1066 s) — the box the packing rule treats as spare capacity is the busiest by count and the slowest to start. And intel is the slowest per job (p95 4994 s).

The correction §1 needs

The first run of the report falsified the spec's own equation. §1 computes max PRs/train ≈ 3 × 72 h / p95(ci / gate). The required checks on main are gate and workspace-test, and a PR merges when the slowest one is green:

required check n p95 total_s
workspace-test 147 6451 ← binds
ci / gate 163 1205
gate 158 255

So §1 as written answers 645 and the honest answer is 120 — a 5.4× overstatement, on the one number §8 uses to decide whether to stop cutting trains ("measured max PRs/train < 10 [A]"). The report computes the bound over the required-check set, names which check binds, and prints the ratio by which §1's own form overstates. The gate: line keeps its specified shape; only the number is honest.

Both spellings are listed because they are one check: branch protection names ci / gate, ruleset 13878864 names a bare gate, and scripts/pr_review_quorum_arm.sh already accepts either for that reason.

Contract

scripts/check_build_report.sh, 35 assertions, run from make lint-scripts and from the new workflow:

  • exit 2 + decline: below the §8 floor of 20 job records — a report over a thin ledger that prints 0 reads exactly like a fast fleet
  • exit 1 + reject: on anything that is not a JSON object, never a silent skip
  • a valid record with no total_s is skipped and the skip is printed (8 of 1092: 5 fleet-pack, 2 reconcile, 1 train)
  • nearest-rank percentiles, case-tabled over 1..100 at k=1, 7, 29, 50, 95, 100 and over [7] and [1,2,3]

The ledger is per job, not per test target; the output says so where §5 asks for "10 slowest test targets" rather than inventing per-target numbers. peak_rss_mb and free_disk_gb are null on every record because the Actions REST API does not expose them — the report prints them as [U], it does not drop them.

Review found four defects, and they are fixed

make gate and a 3-lane review quorum ran against the first green implementation.

# Found by Defect
1 3/3 lanes, grounding=measured a JSON value that is not an object aborted jq with exit 5, not the contracted exit 1. The guard had only tried plain text, which jq rejects cleanly — so the contract looked kept
2 3/3 lanes an empty or whitespace-only file parsed to nothing and exited 0 — silently dropped, never counted
3 1/3 lanes ceil(k/100.0 × n) in floating point: (7/100.0)*100 is 7.000000000000001, so p7 of 1..100 answered 8. The same lane correctly judged that a k=50/95/100 table cannot reach it — too coarse, not wrong
4 check_guards_are_wired.sh build_report.sh was named by no workflow. make lint-scripts runs its guard, but Makefile-only means make tier3, which CI does not run — the same gap toolchain-ceiling.yml documents for check_msrv.sh. Unwired guards grew 3 → 4

Each has a case-table row that goes RED without its fix. Defect 4 is why .github/workflows/build-report.yml exists; the guard is back to its 3-entry baseline.

Gate

make gate reports 4 failures on this branch. None is caused by this diff — each was run on this branch and on a branch without these changes, in the same minute:

check this branch without these changes verdict
check_baseline_ratchets rc=1 rc=1 pre-existing: baselines recorded under pmat 3.39.0 / bashrs 7.0.1, runner has 3.40.0 / 7.4.1
check_complexity_ratchet rc=1 rc=1 pre-existing, same instrument drift
check_silicon_coverage rc=0 now, rc=1 15 min earlier rc=0 time-dependent — the verdict changed with no commit between. A guard whose answer moves on the wall clock is worth its own ticket
check_guards_are_wired rc=0 rc=0 was mine (3 → 4), fixed

How it composes with #3207

#3207 is the writerscripts/collect_build_ledger.sh plus the nightly that appends records. This is the reader. Disjoint files; neither blocks the other. #3207 is what keeps this report's input growing; without it the ledger stops at 1092.

no-close: P0·Instrument has no issue of its own — the row is named in docs/specifications/APR-RELEASE-001-train-and-build-kaizen.md §5, and arrival already outruns closure 2:1, so this files nothing new.

ont-delta: none — no entity type, shape or resolves: target yet; the §11.1 delta for this surface is a code contract over the ledger record schema, and it waits on ONT-1 (pv census), epic #3269.

🤖 Generated with Claude Code

noahgift and others added 4 commits September 14, 2026 15:35
…not exist

APR-RELEASE-001 §5 P0·Instrument's Done test is "≥ 20 records; `make
build-report` runs on a clean checkout". Measured on main: 1092 ledger
records are committed under docs/build-ledger/ in exactly the P0·Instrument
schema (321 of them `ci / gate`), and NOTHING in the tree reads them. So the
first half passes 54× over and the second half has no mechanism at all —
which is why §1's p95, §1's max-PRs-per-train and §7's whole `gate:` line
are still [U] while the data sits in git.

This is the sccache class: recorded 18,292 times, never compared.

The guard asserts seven things, and fails on the first today:
reader exists · has a make-target caller · the self-test has a caller in
lint-scripts (a facility with a self-test and no caller is measured by
nothing) · an empty ledger exits 2 with `decline:` and 19 records still do,
because a report over a thin ledger that prints 0 reads exactly like a fast
fleet · a malformed record exits 1 with `reject:`, never a silent skip ·
the committed ledger reports.

RED: `bash scripts/check_build_report.sh` → exit 1, "scripts/build_report.sh
is absent".

Pmat-Ticket: PMAT-3225
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…hing read

APR-RELEASE-001 §5 P0·Instrument. scripts/build_report.sh streams
docs/build-ledger/, and prints p50/p95 total_s and queue_wait_s per
host_class, the 10 slowest jobs, and §1's PRs-per-train number. 0.27 s over
1092 records, byte-identical across runs.

Contract, asserted by scripts/check_build_report.sh (16/16):
- exit 2 + `decline:` below the §8 floor of 20 job records — a report over a
  thin ledger that prints 0 reads exactly like a fast fleet
- exit 1 + `reject:` on a parse error, never a silent skip
- a valid record with no total_s is SKIPPED as "not a job record" and the
  skip is printed (8 of 1092: 5 fleet-pack, 2 reconcile, 1 train)
- nearest-rank percentiles, case-tabled in --self-test over 1..100 and [7]
- `make lint-scripts` runs the guard, so the self-test has a caller

The ledger is per JOB, not per test target; the output says so where §5 asks
for "10 slowest test targets" rather than inventing per-target numbers.

Verified independently: a separate python implementation reproduces
valid=1092 job=1084 skipped=8 and every p50/p95 exactly.

Pmat-Ticket: PMAT-3225
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…d check

The first run of `make build-report` against the committed ledger falsified
APR-RELEASE-001 §1's own equation.

§1 computes `max PRs/train ~= 3 x 72h / p95(ci / gate)`. The required checks
on `main` are `gate` AND `workspace-test`, and a PR merges when the SLOWEST
of them is green. Measured over 1092 records:

  workspace-test   n=147   p95 6451 s   <- binds
  ci / gate        n=163   p95 1205 s

So §1 as written answers 645 PRs/train and the honest answer is 120 — an
overstatement of 5.4x, on the one number §8 uses to decide whether to stop
cutting trains ("measured max PRs/train < 10 [A]").

The report now computes the bound over the required-check set, prints the
per-check p95 table, names which check binds, and — when the binding check
is not `ci / gate` — prints the ratio by which §1's own form overstates.
The §7 `gate:` line keeps its specified shape; only the number is honest.

RED first: a fixture of 20 fast `ci / gate` (100 s) + 20 slow
`workspace-test` (1000 s) must answer 777, not 7776. The guard's first
extractor read the `3` out of "3 x 72h" and reported got=3 — a guard regex
wrong on its first outing, caught by its own case table, now anchored on the
pinned `gate:` line.

Pmat-Ticket: PMAT-3225
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`make gate` and a 3-lane review quorum ran against the first green
implementation. Between them they found four real defects; all four are
fixed here and each has a case-table row that goes RED without the fix.

1. A JSON value that is not an object aborted jq with exit 5, not the
   contracted exit 1 + `reject:`. `has("total_s")` over an array, number,
   string or boolean is a fatal jq error. Found by 3/3 lanes independently,
   grounding=measured. The guard had only tried plain text, which jq rejects
   cleanly — so the contract looked kept.
2. An empty or whitespace-only file parsed to nothing, emitted nothing, and
   exited 0: silently dropped, never counted as reject or skip. With >= 20
   other records the run reported success over a ledger it had quietly
   discarded part of. Also 3/3 lanes.
3. `ceil(k/100.0 * n)` in floating point: (7/100.0)*100 is
   7.000000000000001, so p7 of 1..100 answered 8. Integer `((k*n)+99)/100`
   is exact. Found by 1/3 lanes, which also correctly judged that the
   k=50/95/100 table could not reach it — the table was too coarse, not
   wrong. It now carries k=1, 7, 29 and a 3-element vector, plus a
   `--percentile-probe` mode the guard reads.
4. `build_report.sh` was named by no workflow — `make lint-scripts` runs its
   guard, but Makefile-only means `make tier3`, which CI does not run.
   check_guards_are_wired.sh caught it (unwired guards 3 -> 4) and is right:
   that is the same gap toolchain-ceiling.yml documents for check_msrv.sh.
   .github/workflows/build-report.yml gives it a real caller; the guard is
   back to its 3-entry baseline.

Also: the required-check set now lists BOTH spellings of the gate. Branch
protection names `ci / gate`, ruleset 13878864 names a bare `gate`, and the
ledger carries both (n=163 p95 1205 s, and n=158 p95 255 s) —
scripts/pr_review_quorum_arm.sh accepts either for the same reason. The
binding check is unchanged (workspace-test, 6451 s) but the table is now
complete rather than missing a third of the gate population.

35/35 guard, 22/22 self-test, byte-identical across runs, bashrs 0 errors.

Pmat-Ticket: PMAT-3225
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@noahgift noahgift added this to the 0.69.0 milestone Sep 14, 2026
@noahgift
noahgift enabled auto-merge September 14, 2026 14:04
…defects

The §0 selector evaluation that picked this row (0: no intel pressure by the
packer's own sample; 1: 31 h since v0.67.0, needs 48; 2: P0·Instrument, the
first §5 row with a MISSING MECHANISM rather than a trailing-window Done
test). Dispatch ledger: both subagents hit their turn limits, neither was
resumed — the worker's claim was re-verified directly and the delegate's
three lanes had already written their artifacts to out_dir.

running_peak=1 of 3 slots; denied=1 (goal.sh set refused a second ticket in
one session, by design — the statusline goal stays PMAT-1098).

Gaps named rather than closed: lane-reduce.sh never ran so there is no
reduced consensus artifact; estimate.sh cannot pool this repo because 38 of
43 rows in impl-estimates.jsonl carry no `unit`, which is why basis is
first-run[U] by force and not by novelty; check_silicon_coverage's verdict
moved with no commit between two runs 15 minutes apart.

Verdict PARTIAL(escalate): the mechanism is landed and armed, every defect
the gate and the quorum found is fixed, and two DoD parts stay open — the pv
contract is NotRun behind ONT-1, and the PR is not merged green yet.

Pmat-Ticket: PMAT-3225
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 14, 2026

Copy link
Copy Markdown

§13.11 rung 1 — quorum shadow verdict

S13-SHADOW pr=3271 head=f28f9d295e148371690c3d3a1978b275132e33cb verdict=REFUSE class=Q1 arm_rc=1

Shadow mode: this records a verdict and merges nothing. A refusal
to arm is not a block (§13 adds zero rows to §7) — the pull request is
exactly as green as it was.

@noahgift
noahgift added this pull request to the merge queue Sep 14, 2026
@noahgift

Copy link
Copy Markdown
Contributor Author

This tool just answered a §5 row that the spec still lists as unstarted work

Dogfooding build_report.sh's own data against §5 P2 · Shard across hosts, whose Done test reads: "one PR's gate runs on ≥ 2 hosts; build-report prints p95 queue-wait per host; zero ENOSPC in 10 trains."

Measured over the committed ledger (1084 job records, 119 distinct shas):

clause verdict
one PR's gate runs on ≥ 2 hosts 112 of 119 shas (94%) ran jobs on ≥ 2 host classes; 50 on ≥ 3
build-report prints p95 queue-wait per host this PR — queue p95 intel 1066 yoga 832 gx10 1399
zero ENOSPC in 10 trains trailing-window, belongs to P0 · Reap
03a00ad55 -> ['gx10', 'intel', 'yoga']
03b526185 -> ['gx10', 'intel']
092cf3ea5 -> ['gx10', 'intel']

So P2 is not pending work — the fleet has been sharding gates across hosts for 119 shas and nothing read the record that proves it. That is the same shape as the row this PR closes: the data was there, nothing compared it.

Worth noting for whoever plans the next build row: §5's queue should be re-derived from the Done tests, not worked top-to-bottom. P0 · Instrument was two-thirds satisfied before this PR (321 gate records ≫ the floor of 20); P2 is two-thirds satisfied now. A row's position in the list is not evidence about its state.

🤖 Generated with Claude Code

@noahgift
noahgift removed this pull request from the merge queue due to a manual request Sep 14, 2026
@noahgift
noahgift added this pull request to the merge queue Sep 14, 2026
@noahgift
noahgift removed this pull request from the merge queue due to a manual request Sep 14, 2026
@noahgift
noahgift added this pull request to the merge queue Sep 14, 2026
@noahgift
noahgift removed this pull request from the merge queue due to a manual request Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant