Skip to content

feat(ladder): a receipt names the binary that ran, never the checkout HEAD; the judge requires the release binary; GPU calls join gpu-q (#3771) - #3784

Closed
noahgift wants to merge 4 commits into
fix/0691-bump-fixture-libsfrom
PMAT-3771-binary-identity
Closed

noahgift wants to merge 4 commits into
fix/0691-bump-fixture-libsfrom
PMAT-3771-binary-identity

Conversation

@noahgift

Copy link
Copy Markdown
Contributor

#3771: a model-ladder receipt names the binary that ran, never the checkout HEAD; the judge requires the release binary

Stacked on release/0.69.1-batch-1 + the fixture-libs fix (42ae22a) + A2 (#3766). The base is fix/0691-bump-fixture-libs, so this diff is A2 + #3771. The quorum judges only #3771's two commits (--base = A2's head on this branch).

Found by aprender-fd: model_ladder.sh recorded sha = the worktree HEAD, and check_model_ladder.sh gated on the version string alone. A run recorded bfd99757c while executing b89d126fc.

Commit 1: the receipt's identity is the binary

file change
scripts/model_ladder.sh sha = the git sha parsed from "$APR" --version; a binary naming no sha is a decline. Also records apr_version verbatim, apr_sha = the binary's full commit, and checkout_sha = HEAD (recorded, never judged). Adds --identity-probe
scripts/check_model_ladder.sh per host: apr_version must carry a sha, sha must equal it, and it must be prefix-equal to the release binary's sha. That sha is read from the pinned binary (apr_bin.sh), or from MODEL_LADDER_RELEASE_SHA for case tables; the verdict names the source. A version-string match alone is refused
scripts/check_release_bump_pr_body.sh fixture receipts name apr 9.9.9 (0123abc), and the fixture run exports MODEL_LADDER_RELEASE_SHA=0123abc
contract MCL-INV-010, FALSIFY-MCL-018

Sweep (done_when 4). Only model_ladder.sh recorded HEAD as the identity of what ran. The rest fall into three groups:

  • pinned by construction: perf002/perf041 probes and dogfood.sh (via apr_bin.sh's freshness check, APR_BIN included);
  • HEAD compared against the release commit: release/models_t1.sh, autopilot.sh, publish_strict.sh, check_publish_preflight.sh;
  • not a measurement's identity: a signing or source repo's own HEAD, git plumbing, and fixtures.

The full per-line list is in the fragment.

Commit 2: ordered GPU access (cop rule rev 6)

apr_locked joins the fleet's gpu-q queue when gpu-q --caps lists wait: GPUQ_WAIT=$LOCK_WAIT gpu-q --prio ${MODEL_LADDER_GPU_PRIO:-1}. Otherwise it keeps the bounded flock, which says it may have jumped the queue. The bounded-wait contract is the same on both routes.

Measured on this head

  • check_model_ladder.sh --self-test: 58 cases, 0 bad. Newly killed: judge mutants binary-mismatch and no-binary-named; producer mutants identity-is-head, gpuq-unbounded and gpuq-no-caps.
  • Probes:
    • a fake apr 9.9.9 (feedbee12) → sha=feedbee12 with checkout ≠;
    • a bounded fake gpu-q → prio 1, wait=7, under the lock at oom 1000;
    • an unbounded gpu-q → not used.
  • check_release_bump_pr_body.sh: 12/12 · contracts lib 1701/0 · pv lint PASS · bashrs 0 errors.

Where the gate runs

check_model_ladder.sh is declared only in Cargo.toml [package.metadata.dogfood] (T-2) and listed in scripts/unwired_guards_baseline.txt, so no PR workflow runs it.

Refs #3771 #3712
keep-open: #3771 closes when this folds; #3712 stays open for its remaining rows.

🤖 Generated with Claude Code

noahgift and others added 4 commits September 21, 2026 21:19
…r, never its name; the judge recomputes membership

quantization". Row A's inventory was a *q4_k*/*q4k* filename glob. #3763 (row A2).

- contract: inventory.patterns -> candidates [*.gguf, *.apr] (by extension) +
  member_dtype Q4_K; MCL-INV-009, FALSIFY-MCL-017.
- model_ladder.sh: every candidate is read with `apr tensors --json` through apr_header
  (no GPU visible, no fleet lock); a member iff Q4_K is a most-frequent dtype of its
  >= 2-D tensors (ties are members; by count, since a Q4_K_M file is Q6_K by bytes);
  the receipt carries every candidate's histogram or its read error.
- check_model_ladder.sh: refuses `patterns`; requires the candidates; names an
  unreadable one; recomputes membership and FAILs a Q4_K file left out of the
  inventory or a non-Q4_K file let in.
- case table 47 -> 55; judge mutants filename-glob, header-excluded, header-included,
  tie-not-member and producer mutant header-sees-gpu are each killed.

Measured dry run on lambda: f16, IQ4_XS and an f32 .apr are not members; the Q4_K_M
GGUF, the q4k .apr and the Q4_K_M file renamed unlabelled-model.gguf are.

Refs #3763 #3712

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ucer, judge and #3742

aprender-c3's #3742 gate must use the same universe as the ladder, not a second copy,
and the judge's recomputation was itself a second copy. scripts/lib/tensor_universe.py
now holds the rule (>= 2-D tensor dtypes by count, upper-cased; dominant = every dtype
at the max; member iff the dtype is among them, ties included; the name never reaches
it). The producer calls its `row` CLI on the header it read through apr_header; the
judge imports is_member. scripts/lib/tensor_universe.sh (sourced, option-neutral) gives
shell gates tu_dominant_dtypes / tu_is_member, reading with CUDA_VISIBLE_DEVICES="".

The definition's own 7-case table (incl. a tie is a member, 1-D tensors do not vote, a
Q5_0-dominant file is not a member whatever its name) runs in check_model_ladder.sh
--self-test; a mutant of the shared rule is killed by red-tie-excluded. Measured via the
shell wrapper on lambda: Q4_K_M gguf and q4k .apr -> member; IQ4_XS, f32 .apr -> not;
a missing file -> rc 2. `CUDA_VISIBLE_DEVICES= cmd` is spelled `=""` (SC1007).

Refs #3763 #3742

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…, never the checkout HEAD; the judge requires the release binary

#3771 (found by aprender-fd): model_ladder.sh recorded `sha` = the worktree HEAD while the
binary's own identity went to `apr_version`, which check_model_ladder.sh never read; it
gated on the version STRING alone. A run recorded bfd99757c while executing b89d126fc.

- model_ladder.sh: `sha` = the git sha parsed from `"$APR" --version` (a binary naming no
  sha is a decline); `apr_version` verbatim; `apr_sha` = the BINARY's full commit;
  `checkout_sha` = HEAD, recorded and never judged. --identity-probe for the table.
- check_model_ladder.sh: per host, apr_version must carry a git sha, `sha` must equal it,
  and it must be prefix-equal to the RELEASE binary's sha -- read from the pinned binary
  (scripts/apr_bin.sh), or MODEL_LADDER_RELEASE_SHA for case tables; the verdict names
  the source. Cases red-binary-sha-differs-same-version, red-receipt-head-only,
  red-receipt-sha-is-not-its-binary; mutants binary-mismatch, no-binary-named,
  identity-is-head each killed. MCL-INV-010, FALSIFY-MCL-018.
- check_release_bump_pr_body.sh: fixture receipts name `apr 9.9.9 (0123abc)`; the
  prepare_bump run exports MODEL_LADDER_RELEASE_SHA=0123abc.
- sweep (done_when 4): only model_ladder.sh recorded HEAD as the identity of what ran; the
  perf probes and dogfood.sh are pinned by construction; the rest compare HEAD to a
  release commit, read a repo's own HEAD, or are git plumbing and fixtures (fragment).

Refs #3771 #3712

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… it bounds its wait

Cop rule rev 6: `gpu-q` orders GPU access by (priority, arrival) in front of the same
flock + choom, and v3 bounds the whole wait (GPUQ_WAIT; exit 75 naming the holder). The
ladder's apr_locked flocked /tmp/apr-gpu.lock directly: it excluded correctly but could
jump the queue.

- model_ladder.sh: when gpu-q is present and `gpu-q --caps` lists `wait`, apr_locked runs
  `GPUQ_LOCK=$GPU_LOCK GPUQ_WAIT=$LOCK_WAIT gpu-q --prio ${MODEL_LADDER_GPU_PRIO:-1} -- ...`
  (1 = release-blocking); otherwise the bounded flock, which says it may have jumped the
  queue. The bounded-wait contract is unchanged on both routes. The run header prints the
  route; MODEL_LADDER_GPU_Q is a test seam (empty = never gpu-q).
- check_model_ladder.sh: the existing lock probes pin the bare route (never the fleet
  queue); two new probes drive a bounded fake gpu-q (prio 1, GPUQ_WAIT = the lock wait,
  under the lock at oom 1000) and an unbounded one (not used, fallback says so); producer
  mutants gpuq-unbounded and gpuq-no-caps are each killed. FALSIFY-MCL-015 updated.

Refs #3771

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@noahgift

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3771",
 "head": "b69348396e1cef4a24740e7a71edb4cc51cb2504",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "PASS",
   "findings": 10
  },
  {
   "lane": 2,
   "verdict": "NO-VERDICT",
   "findings": 0
  },
  {
   "lane": 3,
   "verdict": "PASS",
   "findings": 0
  }
 ]
}

@noahgift

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3771",
 "head": "b69348396e1cef4a24740e7a71edb4cc51cb2504",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "PASS",
   "findings": 14
  },
  {
   "lane": 2,
   "verdict": "NO-VERDICT",
   "findings": 0
  },
  {
   "lane": 3,
   "verdict": "NO-VERDICT",
   "findings": 0
  }
 ]
}

@noahgift

Copy link
Copy Markdown
Contributor Author

Seat status (aprender-62, 02:34Z): NOT seated, and it needs 2 fills. Head b69348396, judged against 395756d06 (A2's head). There is 1 independent PASS, gpt-oss in lane 3 of quorum-archive/3771-gemini-503-022336/. There are also 2 claude-opus-4-6-thinking PASSes, but those are same family as the author and don't count as independent. Every other lane returned no verdict (503). Self-test proof: check_model_ladder.sh --self-test gives 58 cases, 0 bad; guard_tree --no-cargo gives 92/0; check_release_bump_pr_body.sh gives 12/12.

@noahgift

Copy link
Copy Markdown
Contributor Author

Triage #4158 (needs-owner): its base fix/0691-bump-fixture-libs is a branch with no PR: retarget to main. The cop assigns an owner from the 0.70 scope.

@noahgift

Copy link
Copy Markdown
Contributor Author

fold source for the 0.70 integration branch; branch kept at PMAT-3771-binary-identity @ b69348396e1cef4a24740e7a71edb4cc51cb2504.

Closed under the operator 1-day PR rule (2026-09-24); the rule does not wait on #4046. aprender-6c folds from the branch ref above into the 0.70 integration branch (built off main after #4046 merges).

@noahgift noahgift closed this Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind:code Work is a code change (derived rule, #4159) needs-owner Open work with no live owner — 0.70 batch triage; pick it up and reassign

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant