feat(ladder): Q4_K universe read from each file's tensor header, not its name; judge recomputes membership (#3712 row A2) by noahgift · Pull Request #3766 · paiml/aprender

noahgift · 2026-09-21T18:31:08Z

#3712 row A2: the Q4_K universe is read from each file's tensor header, never its name; the judge recomputes membership

Stacked on #3738 (row B1, which is stacked on #3721). The base is PMAT-3712-cells, so this diff is A2 only.

The #3712 issue author clarified done_when 1: "filename is not evidence of quantization". Row A's inventory was a *q4_k*/*q4k* filename glob, and this replaces it.

What changes

file	change
`contracts/model-capability-ladder-v1.yaml`	`inventory.patterns` (the glob) → `candidates: [.gguf, .apr]` (by extension) + `member_dtype: Q4_K`; MCL-INV-009; FALSIFY-MCL-017
`scripts/model_ladder.sh`	reads EVERY candidate with `apr tensors --json` through `apr_header` (no GPU visible, and no fleet lock, so a held lock can't stall the inventory); counts the dtypes of its ≥2-D tensors; a member iff Q4_K is a most-frequent dtype (ties are members); the receipt carries every candidate's histogram, or its read error
`scripts/check_model_ladder.sh`	refuses a ladder that still names `patterns`; requires the candidates list; names an unreadable candidate; recomputes membership from each histogram, and FAILs a Q4_K-header file left out of the inventory and a non-Q4_K file let into it
`scripts/lib/tensor_universe.py` + `.sh`	ONE definition of the universe, shared by the producer, the judge (its private copy is gone) and #3742: histogram of ≥2-D dtypes by count, dominant = every dtype at the max, member iff the dtype is among them. Shell gates get `tu_dominant_dtypes` / `tu_is_member` (sourced, option-neutral, no GPU visible). Its own 7-case table runs in the self-test, and a mutant of the shared rule is killed

Why by count, and why `apr tensors`

Measured on lambda:

apr inspect --json's quantization says Q6_K for Qwen3.5-0.8B-Q4_K_M.gguf, None for every .apr, and inspect fails outright on IQ4_XS. That defect is filed as apr inspect --json 'quantization' is wrong: Q6_K for a Q4_K_M GGUF, None for every .apr, and inspect FAILS on IQ4_XS ('Unsupported dtype 23') #3762 and is not fixed here.
By bytes, the same Q4_K_M file is Q6_K, because of its 248k-vocab embedding/output. By count of ≥2-D tensors it is Q4_K: 98 Q4_K, 36 Q5_K, 36 Q8_0, 18 F32, 17 Q6_K.

Dry run on real lambda files:

file	read as	member?
`Qwen2.5-0.5B-Instruct-f16.gguf`	F16	no
`Qwen3.5-0.8B-IQ4_XS.gguf`	IQ4_XS	no
`Qwen3.5-0.8B-Q4_K_M.gguf`	Q4_K	yes
`qwen2.5-coder-1.5b-instruct-q4k.apr`	Q4_K	yes
`qwen2.5-coder-1.5b-instruct-st.apr`	F32	no
the Q4_K_M file symlinked as `unlabelled-model.gguf`	Q4_K	yes

Measured on this head

check_model_ladder.sh --self-test: 55 cases, 0 bad (47 → 55).
Every mutant is killed: 7 judge (including the 4 new A2 ones: filename-glob, header-excluded, header-included, tie-not-member), 5 producer (including header-sees-gpu), and 11 cells.
pv validate ok · pv lint contracts/ PASS · census and contracts.nt unchanged · bashrs 0 errors · shell-lint ratchet PASS.

Where the gate runs

check_model_ladder.sh is declared only in Cargo.toml [package.metadata.dogfood] (the T-2 pre-publish dogfood). It is also listed in scripts/unwired_guards_baseline.txt: no PR workflow runs it. Its real run is RED until a cut has receipts. That is the release bar, not a PR red.

Refs #3763 #3712
keep-open: #3763 closes when this row folds. #3712 stays open for its remaining rows. #3762 is a separate fix.

🤖 Generated with Claude Code

…r, never its name; the judge recomputes membership #3712 done_when 1, clarified by the issue author: "filename is not evidence of quantization". Row A's inventory was a *q4_k*/*q4k* filename glob. #3763 (row A2). - contract: inventory.patterns -> candidates [*.gguf, *.apr] (by extension) + member_dtype Q4_K; MCL-INV-009, FALSIFY-MCL-017. - model_ladder.sh: every candidate is read with `apr tensors --json` through apr_header (no GPU visible, no fleet lock); a member iff Q4_K is a most-frequent dtype of its >= 2-D tensors (ties are members; by count, since a Q4_K_M file is Q6_K by bytes); the receipt carries every candidate's histogram or its read error. - check_model_ladder.sh: refuses `patterns`; requires the candidates; names an unreadable one; recomputes membership and FAILs a Q4_K file left out of the inventory or a non-Q4_K file let in. - case table 47 -> 55; judge mutants filename-glob, header-excluded, header-included, tie-not-member and producer mutant header-sees-gpu are each killed. Measured dry run on lambda: f16, IQ4_XS and an f32 .apr are not members; the Q4_K_M GGUF, the q4k .apr and the Q4_K_M file renamed unlabelled-model.gguf are. Refs #3763 #3712 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

noahgift · 2026-09-21T18:40:26Z

quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3763",
 "head": "36f77e0d6a440bab4c38e67597b4b4cead98b00e",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "PASS",
   "findings": 16
  },
  {
   "lane": 2,
   "verdict": "NO-VERDICT",
   "findings": 0
  },
  {
   "lane": 3,
   "verdict": "NO-VERDICT",
   "findings": 0
  }
 ]
}

noahgift · 2026-09-21T18:45:03Z

quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3763",
 "head": "36f77e0d6a440bab4c38e67597b4b4cead98b00e",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "PASS",
   "findings": 24
  },
  {
   "lane": 2,
   "verdict": "NO-VERDICT",
   "findings": 0
  },
  {
   "lane": 3,
   "verdict": "NO-VERDICT",
   "findings": 0
  }
 ]
}

…ucer, judge and #3742 aprender-c3's #3742 gate must use the same universe as the ladder, not a second copy, and the judge's recomputation was itself a second copy. scripts/lib/tensor_universe.py now holds the rule (>= 2-D tensor dtypes by count, upper-cased; dominant = every dtype at the max; member iff the dtype is among them, ties included; the name never reaches it). The producer calls its `row` CLI on the header it read through apr_header; the judge imports is_member. scripts/lib/tensor_universe.sh (sourced, option-neutral) gives shell gates tu_dominant_dtypes / tu_is_member, reading with CUDA_VISIBLE_DEVICES="". The definition's own 7-case table (incl. a tie is a member, 1-D tensors do not vote, a Q5_0-dominant file is not a member whatever its name) runs in check_model_ladder.sh --self-test; a mutant of the shared rule is killed by red-tie-excluded. Measured via the shell wrapper on lambda: Q4_K_M gguf and q4k .apr -> member; IQ4_XS, f32 .apr -> not; a missing file -> rc 2. `CUDA_VISIBLE_DEVICES= cmd` is spelled `=""` (SC1007). Refs #3763 #3742 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…efinition (#3742, #3712) The gate printed a dominant dtype by BYTES and judged UNCOVERED by it; bytes evict the Qwen3.5 0.8B/2B Q4_K_M rungs (their tied Q6_K embedding outweighs the layers). The cop ruled the definition (most frequent >= 2-D tensor dtype by COUNT, ties as members) and ONE implementation: scripts/lib/tensor_universe.{sh,py}, taken verbatim from A2 (#3766, PMAT-3763-a2-header-universe @ 3e69272), which folds before this row. The gate's MODEL line carries `dtype=` (tu_dominant_dtypes) and `q4k-universe=` (tu_is_member); a pair that cannot be compared is UNCOVERED (RED) only for a member, OUTSIDE otherwise. The gate's own byte-based helper and its case rows are gone; the rule is proven in the lib's own table. Measured: Qwen3.5-0.8B-Q4_K_M.gguf dtype=Q4_K member; qwen2.5-coder-1.5b q4k.apr Q4_K member (paired with the q4_k_m GGUF by fingerprint); the -st.apr F32 OUTSIDE. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

noahgift · 2026-09-22T02:17:17Z

quorum-review (AD-04): three PASS — agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3763",
 "head": "3e69272a13af9efbab73ab982aa41ab51e6e8333",
 "width": 3,
 "executor": "agy",
 "agreed": true,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "PASS",
   "findings": 0
  },
  {
   "lane": 2,
   "verdict": "PASS",
   "findings": 0
  },
  {
   "lane": 3,
   "verdict": "PASS",
   "findings": 0
  }
 ]
}

noahgift · 2026-09-22T02:35:04Z

Seat status (aprender-62, 02:34Z): the 02:17Z round is "AGREED 3/3" and receipt-lint clean. But the precheck substituted gpt-oss-120b-medium into all three lanes, so that's one distinct model id, and the summaries are thin ("No refutations found"). Under the cop's distinct-id rule it's one seat, so it needs 2 fills or a ruling. Artifact: docs/audits/quorum-PMAT-3763.json in the author's worktree. Self-test proof: check_model_ladder.sh --self-test gives 55 cases, 0 bad; guard_tree --no-cargo gives 76/0.

Row A2 (the Q4_K universe read from each file's tensor header). The receipt is committed on its own, after the head it judged, so the judged diff is unchanged. Honest reading of its partial_reasons: all three lanes ran on ONE model id (gpt-oss-120b-medium), because the gemini family was quota-exhausted (429 on the pre-check probe) and gpt-oss itself 503'd twice before answering. Under PMAT-125 that is agreement under resampling, not three independent seats, so this row was queued for two peer seats. Those seats are suspended by the wind-down and were never filled: treat this as ONE seat, not three. Pmat-Ticket: PMAT-3763 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…tch-1 without them Both rounds AGREED 3/3 PASS and both artifacts existed only as untracked files in my worktrees; the branches they judged are already ancestors of release/0.69.1-batch-1, so the evidence would have died with the worktree while the code it cleared shipped. quorum-PMAT-3712-rowA.json head 6eee66b, base origin/main (row A: the ladder's model inventory) quorum-PMAT-3712-b1.json head fd636b3, base origin/PMAT-3712-ladder-universe-inventory (row B1: the cells judge) Two files rather than one quorum-PMAT-3712.json because they are two rounds over two different diffs of the same ticket; the existing quorum-PMAT-3477-pr*.json pair is the precedent for the suffix. rowA carries one partial_reason (lane 3's grounding could not be compared, PMAT-082); b1 carries none. Pmat-Ticket: PMAT-3712 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

noahgift · 2026-09-24T06:47:14Z

Triage #4158 (needs-owner): its base PMAT-3712-cells is the head of CLOSED PR #3738: retarget to main. The cop assigns an owner from the 0.70 scope.

noahgift · 2026-09-24T07:58:30Z

fold source for the 0.70 integration branch; branch kept at PMAT-3763-a2-header-universe @ 7e895c611dac2809db5f1a25ac8e84497f6810c5.

Closed under the operator 1-day PR rule (2026-09-24); the rule does not wait on #4046. aprender-6c folds from the branch ref above into the 0.70 integration branch (built off main after #4046 merges).

noahgift mentioned this pull request Sep 21, 2026

feat(ladder): a receipt names the binary that ran, never the checkout HEAD; the judge requires the release binary; GPU calls join gpu-q (#3771) #3784

Closed

noahgift mentioned this pull request Sep 22, 2026

release/0.69.1-batch-1: check_release_bump_pr_body.sh fixture ladder receipts are the pre-#3712 shape — the real judge no longer passes them (combination break of #3721 + #3738) #3770

Closed

noahgift and others added 2 commits September 22, 2026 05:21

noahgift added kind:code Work is a code change (derived rule, #4159) needs-owner Open work with no live owner — 0.70 batch triage; pick it up and reassign labels Sep 24, 2026

noahgift closed this Sep 24, 2026

Navigation Menu

feat(ladder): Q4_K universe read from each file's tensor header, not its name; judge recomputes membership (#3712 row A2) - #3766

feat(ladder): Q4_K universe read from each file's tensor header, not its name; judge recomputes membership (#3712 row A2)#3766
noahgift wants to merge 4 commits into
PMAT-3712-cellsfrom
PMAT-3763-a2-header-universe

noahgift commented Sep 21, 2026 •

edited

Loading

Uh oh!

noahgift commented Sep 21, 2026

Uh oh!

noahgift commented Sep 21, 2026

Uh oh!

noahgift commented Sep 22, 2026

Uh oh!

noahgift commented Sep 22, 2026

Uh oh!

noahgift commented Sep 24, 2026

Uh oh!

noahgift commented Sep 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

noahgift commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

#3712 row A2: the Q4_K universe is read from each file's tensor header, never its name; the judge recomputes membership

What changes

Why by count, and why apr tensors

Measured on this head

Where the gate runs

Uh oh!

noahgift commented Sep 21, 2026

Uh oh!

noahgift commented Sep 21, 2026

Uh oh!

noahgift commented Sep 22, 2026

Uh oh!

noahgift commented Sep 22, 2026

Uh oh!

noahgift commented Sep 24, 2026

Uh oh!

noahgift commented Sep 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

noahgift commented Sep 21, 2026 •

edited

Loading

Why by count, and why `apr tensors`