feat(PMAT-1095): 0.67 CUDA Rust fleet readiness — fix the two wrong-host GPU tests, add scripts/cuda_rust_fleet_check.sh with per-host receipts (gx10 PASS, yoga PASS, lambda-vector blocked on a reboot) by noahgift · Pull Request #3068 · paiml/aprender

noahgift · 2026-09-09T15:10:36Z

2>&1 | tail -1
PMAT-1095 · epic #3062 · dark-modules issue #3067 · spec #3061. Run through paiml-implement (Phase 0 admit, three quorum rounds via the agy delegate, Phase 4 gates). Receipt: docs/audits/impl-PMAT-1095-receipt.md.

What it does

test_alloc_oversize_100gb no longer hard-codes a 100 GB / RTX 4090 assumption. It classifies the device: discrete asks cuMemAlloc for 2× the device; unified memory asks for 2⁶⁰ bytes, which fails validation with nothing to back it from. The device allocator is pinned with an RAII EnvVarGuard (restores prior state on unwind).
test_cublas_gemm_f16_training_shape reports throughput instead of asserting a 50 TFLOP/s wall-clock floor (15.4 on GB10 under suite load, 165 on a 4090 alone). Parity stays asserted; the stale FALSIFY-CUBLAS-* citations that named a non-existent contract are gone.
scripts/cuda_rust_fleet_check.sh — a per-host receipt for cuda-core / cutile / aprender-gpu readiness with a three-state verdict (PASS / INCOMPLETE / FAIL: a skipped prerequisite can never read as a pass; zero probes or an unknown status is FAIL), a --self-test case table that also drives the receipt writer, env-death classification (network → SKIP:env), persisted per-probe logs, ssh keepalive, and a resource guard (refuses under a running CI job — checked twice — and below 12 GiB fail-closed; nice'd, capped jobs, build and test under systemd-run MemoryMax=min(48G, 60% of MemAvailable)).

Receipts (committed, `evidence/cuda-rust-fleet/`)

host	GPU	driver	toolkit	verdict
gx10-a5b5	GB10 sm_121	590.48.01	13.3	PASS 6/6
yoga	RTX 4060 Laptop sm_89	595.91.07	13.3	PASS 6/6
noah-Lambda-Vector	RTX 4090 sm_89	570.207	13.3	INCOMPLETE — `blocked_on` = one root entry: driver < R580; an R580+ package is staged in Lambda's repo and needs a reboot the operator owns

What went wrong on the way, owned

The first draft of this runner OOMed gx10 (the only Blackwell CI host): a cold cargo test --features cuda alongside P1 v2's 2×-physical cuMemAlloc on a unified-memory part put the box into global OOM at 13:36 (this test binary at the top of the OOM table); it rebooted at 13:43 and came back with no default route (its DHCP profile is autoconnect=false; restored with nmcli con up, no config change). Every guard above exists because of this.
Installing driver 580 on lambda-vector broke CUDA live — the userspace libs were replaced under the running 570 module. Rolled back to the captured 570 package set; CUDA verified working again. "Activates on reboot" was wrong; "breaks until reboot" is the truth.
Quorum: round 1 returned 3/3 do-not-implement with 8 findings; round 2 left 3 (+ a delegate-verified fail-open on an unreadable MemAvailable); round 3 is 3/3 implement-as-written, agreed=true, no dissent.

Gates

make gate 39/0 · fmt · clippy -D warnings (cuda) · bashrs 0 errors · --self-test PASS · roadmap additive (+25) · kind-gate code. pv: no contract changed here. The present pr-review receipt is NotRun (not a required check).

🤖 Generated with Claude Code

https://claude.ai/code/session_01J9cSQynVPYeUkQ2i7ccvrs

… / lambda-vector) Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J9cSQynVPYeUkQ2i7ccvrs

… not a hard-coded 100 GB / RTX 4090 The test allocated a fixed 25e9 f32 (100 GB) and panicked "impossible on RTX 4090!" if it succeeded. On gx10 -- an NVIDIA GB10 with ~128 GB of UNIFIED memory -- a 100 GB allocation legitimately succeeds, so the test FAILED there. It failed under CUDA 13.0 and 13.3 alike (a control run on 2026-09-09), so it was a wrong-host assumption, not a toolkit regression; it passed on the 4090 only by accident of that card being 24 GB. Now: query CudaContext::total_memory() and request 2x the whole device. That exceeds physical memory on every CUDA device, unified or not, and scales with the host. The adversarial intent -- an allocation larger than the device must fail -- is kept; the card name is gone from the assertion and the panic names the requested and total byte counts instead. A_1 on lambda-vector (RTX 4090): test result ok. gx10 re-run follows in the PMAT-1095 receipt. Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J9cSQynVPYeUkQ2i7ccvrs

…ed memory oversubscribes by design on GB10 Measured on gx10: with the default routing, a 2x-the-device (257 GB vs 128 GB) allocation SUCCEEDED because GpuBuffer::new routes integrated parts to cuMemAllocManaged (PMAT-769), which oversubscribes by design. The falsifiable property is "cuMemAlloc refuses more than the device", so the test now sets MANAGED_MEMORY=0 (under the exclusivity lock that already covers env mutation) and asserts against the device allocator explicitly. Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J9cSQynVPYeUkQ2i7ccvrs

…ead of asserting a 50 TFLOP/s wall-clock floor Measured 2026-09-09 on gx10: 15.4 TFLOP/s inside the full --features cuda suite, passing in isolation -- a wall-clock assertion failing under load, the class already burned four times (NO wall-clock assertion in a required check). Correctness stays asserted via the two assert_eq! on C[0,0] and C[last]; the speed claim belongs to the beat/bench lane. The measurement itself is still asserted valid (finite, > 0). Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J9cSQynVPYeUkQ2i7ccvrs

…t for 2x physical — it OOMed and rebooted gx10 The v2 test asked GB10 (128 GB unified) for a 257 GB cuMemAlloc. On a discrete card that is rejected against the VRAM pool before a page is touched; on a unified-memory part the driver tried to BACK it from system RAM. gx10 went global-OOM at 13:36 on 2026-09-09 with this test binary at the top of the OOM table, and rebooted at 13:43. A run that "passed in 34 s" earlier the same afternoon was that thrash, survived by luck. v3 branches on classify_device_memory(): discrete keeps 2x the device; UnifiedMemory asks for 2^60 bytes, which fails validation with nothing to back it from. The same property is asserted -- larger than the device must fail. memory/mod.rs re-exports classify_device_memory + DeviceMemoryClass so tests can reach the classifier. Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J9cSQynVPYeUkQ2i7ccvrs

…NVIDIA CUDA Rust readiness, with a verdict that can fail P3 of PMAT-1095. Turns "0.67 works with cuda-core / cutile on gx10, yoga, lambda-vector" from a sentence into a receipt per host, evidence/cuda-rust-fleet/<host>.json. Six probes, each PASS | FAIL | SKIP:<reason>: driver_floor >= R580 (cuda-bindings refuses older at run time; measured) toolkit_floor >= 13.1 for sm_100+, >= 13.2 for sm_8x (cutile) clang_resource_dir bindgen needs clang's OWN stddef.h; a bare libclang is not enough (measured on yoga: "'stddef.h' file not found"); discovered like cuda-oxide does — clang, then clang-N under /usr/lib/llvm-N/bin (gx10 has only clang-21, invisible to a non-interactive PATH) cudacore_probe experiments/cuda-rust-probes/cudacore: PTX load + launch + readback cutile_probe experiments/cuda-rust-probes/cutile: Tile kernel JIT + run, prints the REAL compute capability (a copied probe once printed "sm_121" on an sm_89 host) aprender_gpu_cuda the tests this ticket and O2/#3066 added or repaired Verdict has THREE states on purpose: PASS only if every probe PASSed; INCOMPLETE when a prerequisite was SKIPped (never counted as a pass — "flawless on three hosts" must not be reachable by skipping the probes that would fail); FAIL on any FAIL, on zero probes (vacuity), or on a status outside {PASS,FAIL,SKIP} (a blacklist is fail-open on its complement). --self-test is a case table over that function; it caught a stale row in its own table on the first run. RESOURCE GUARD, because the first draft of this runner OOMed the only Blackwell CI host. On 2026-09-09 a cold `cargo test -p aprender-gpu --features cuda` here, alongside a test that asked a unified-memory GB10 for 2x physical RAM, put gx10 into global OOM at 13:36 (this test binary at the top of the OOM table) and it rebooted at 13:43. So P6 now: refuses while a GitHub Actions job runs on the host; refuses below 12 GiB MemAvailable (32 was first tried and is unreachable on yoga, a ~31 GiB laptop); nice -n 19 with CARGO_BUILD_JOBS capped; and runs the TEST PROCESS under `systemd-run --scope -p MemoryMax=48G` so a regression can kill only itself. MemAvailable is in the receipt. Environment death is classified, not blamed on code: a network failure (gx10 came back from its reboot with no default route — "Could not resolve host: index.crates.io") retries --offline once and then records SKIP:env, i.e. INCOMPLETE, never PASS and never a FAIL that names a CUDA Rust cause. Every probe's full log is persisted next to the receipt and pulled back from remote hosts; the first draft deleted them and reported "exit=101" with an empty reason. Two runner bugs found by running it: cargo prints the test-binary path RELATIVE to the manifest dir when CARGO_TARGET_DIR is unset (gx10) and absolute when set (lambda-vector, which hid the bug); and a silent ssh session was measured dropping mid-build ("client_loop: send disconnect: Broken pipe"), so --host mode uses ServerAlive keepalives and tees to a remote log. Receipts committed (2026-09-09): gx10-a5b5 PASS GB10 sm_121, driver 590.48.01, CUDA 13.3 — 6/6 yoga PASS RTX 4060 Laptop sm_89, driver 595.91.07, CUDA 13.3 — 6/6 noah-Lambda-Vector INCOMPLETE RTX 4090 sm_89, driver 570.207 < R580 — cuda-core and cutile SKIP; the repaired aprender-gpu tests PASS. 580 is staged in Lambda's repo; taking it needs a reboot the operator owns (installing it live replaced the userspace libs under the running 570 module and broke CUDA until rolled back — measured). The probe crates carry their own [workspace] (verified: cargo metadata --no-deps lists 79 packages, none under experiments/), so --all-features cannot drag cuda-bindings' build.rs into make coverage / make mutants. Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J9cSQynVPYeUkQ2i7ccvrs

…le contract claim, set -e abort path, cgroup sizing, TOCTOU, exact pins, truthful lambda-vector acceptance Phase 3 quorum (3 independent agy lanes, all do-not-implement-as-written) found eight defects; every one is addressed here and each was falsifiable against the export. adversarial.rs - MANAGED_MEMORY was set with std::env::set_var and cleared with an unconditional remove_var: a panic between them leaked the override into every later test in the process, and remove_var destroyed any value the suite was launched with. Replaced by EnvVarGuard, which records the PRIOR state and restores it on drop, including unwind. cublas_tests.rs - The doc comment still claimed "FALSIFY-CUBLAS-003: Must achieve > 100 TFLOP/s". No contract under contracts/ carries that id or any TFLOP/s floor (measured by three lanes and by grep). The comment now says what is true: throughput is reported, no floor exists anywhere, and a speed claim belongs to the beat/bench lane. Coverage did not silently drop -- it never existed outside this one wall-clock assert. scripts/cuda_rust_fleet_check.sh - `bin=$(grep … | sed … | head -1)` under set -euo pipefail: an unmatched grep aborted the whole script BEFORE the receipt was written. Now `|| true`, with an explicit not-found branch. - MemoryMax=48G was a magic number that protected nothing on a host with less than 48 GiB (yoga has ~31). Now min(48 GiB, 60% of MemAvailable), floor 4 GiB, for BOTH the build and the test process. - The pgrep Runner.Worker check was TOCTOU: a CI job could start after it. The build now runs under the same cgroup, and the check is repeated after the build, right before the test runs. - --self-test exercised only the verdict function; the python receipt writer was untested. It is now a function, and the self-test drives it with synthetic probes and asserts the JSON (verdict, probe statuses, typed mem field). That new case immediately caught a real bug: bash arrays cannot ride a command's env prefix (they arrive as the literal string "(a b)"). - Receipts carry `blocked_on`: the SKIP reasons, so an INCOMPLETE names its blocker in the artifact itself rather than in a PR comment. - bashrs: 0 errors (the guarded rm -rf was SEC011). experiments/cuda-rust-probes - cuda-core / cutile pinned exactly (=0.3.1); a diagnostic that drifts with a minor release is not a diagnostic. Cargo.lock committed for both. docs/roadmaps/roadmap.yaml - The lambda-vector acceptance line was contradicted by its own receipt. It now says what is true and still falsifiable: PASS, or INCOMPLETE with driver_floor as the ONLY skipped prerequisite and blocked_on naming the reboot that activates the staged 580 driver. An INCOMPLETE that names its blocker is the truthful artifact; a PASS reached by skipping the probe that would fail is not. Verified on the RTX 4090: both repaired tests pass; clippy -D warnings clean under --features cuda; --self-test PASS; check_roadmap_diff_additive.sh PASS. Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J9cSQynVPYeUkQ2i7ccvrs

…SS, lambda-vector INCOMPLETE (blocked_on: driver 570 < R580, reboot activates staged 580) Re-run with the fixed runner on all three hosts. yoga now PASSes 6/6 with the cgroup sized to 16G on a 28 GiB host (the dynamic MemoryMax from the quorum finding). lambda-vector stays INCOMPLETE and the receipt itself names the blocker in blocked_on. Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J9cSQynVPYeUkQ2i7ccvrs

…ule doc, blocked_on names the reboot and lists root causes only, memory floor fails CLOSED F2: cublas_tests.rs line 6 cited cublas-gemm-v1.yaml (FALSIFY-CUBLAS-001/-003/-005); neither the file nor the ids exist (delegate-verified). It now names contracts/fp16-cublas-gemm-v1.yaml and states no TFLOP/s floor exists. F8: the driver_floor SKIP reason names the remedy (R580+ driver package AND a reboot to activate it) and blocked_on excludes probes skipped BECAUSE of another prerequisite, so lambda-vector lists exactly one root blocker. N1 (new, delegate-measured): with MemAvailable unreadable, awk printed nothing and exited 0, `|| echo 0` never fired, and the 12 GiB floor fell THROUGH to the build -- fail-open. The floor is now a pure function that refuses empty or non-numeric input, self-tested with "", "x", "0", "5", "11" (refuse) and "12", "20", "115" (allow). Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J9cSQynVPYeUkQ2i7ccvrs

…e memory floor glob-free and the self-test loops parseable (bashrs 0 errors) The previous commit pushed with three bashrs errors (BRS0010 on a [!0-9] glob it reads as a test bracket, SC2135 on two one-line for/if loops) and one remaining FALSIFY-CUBLAS-005 doc id that no contract carries. All three fixed; --self-test PASS. Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J9cSQynVPYeUkQ2i7ccvrs

…a-vector INCOMPLETE with one root blocker that names the reboot Re-run on all three hosts with the final runner (sha in each receipt). lambda-vector blocked_on lists exactly one root entry: driver_floor, whose reason states the R580+ package and the reboot that activates it; the two probe skips derived from it are no longer listed as separate blockers. Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J9cSQynVPYeUkQ2i7ccvrs

…ness Identity, routing, dispatch ledger (4 delegate dispatches: ph1 goal partial, ph3 quorum r1 partial / r2 / r3 agreed), claimed-vs-rerun verification table, jidoka log (gx10 OOM+reboot, lambda-vector driver rollback, stray delegate worktree), estimates (K-hat 4 first-run[U], actual ~96), gaps (lambda-vector reboot, pr-review receipts NotRun, vacuous transcript-gate), verdict DONE pending required checks. Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J9cSQynVPYeUkQ2i7ccvrs

…mpty file The earlier writer used an unquoted heredoc whose prose held a backticked apt command; the substitution hung on sudo and produced 0 lines. Written by python now. Estimates row appended. Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J9cSQynVPYeUkQ2i7ccvrs

github-actions · 2026-09-09T17:34:56Z

§13.11 rung 1 — quorum shadow verdict

S13-SHADOW pr=3068 head=ff8727fbc1209f294de403d4e7a652f9c7e0f073 verdict=REFUSE class=Q1 arm_rc=1

Shadow mode: this records a verdict and merges nothing. A refusal
to arm is not a block (§13 adds zero rows to §7) — the pull request is
exactly as green as it was.

…ipped-path ratchet was +4 on the merged tree (guard-tree red on run 34368743080) Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…nions) Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

… and a fleet-neutral VRAM floor: gx10 6 FAILED (906/901 stream-capture vs legacy-stream H2D across concurrent tests), yoga 2 FAILED (">20GB VRAM" on an 8 GB 4060) Measured at bb0b32e inside the disposable-runner image with the host toolkit mounted: gx10 2595/6, yoga 2599/2. A capture opened on a blocking stream makes every legacy-stream transfer in the process fail with CUDA_ERROR_STREAM_CAPTURE_IMPLICIT (906) and invalidates the capture (901); CudaStream::new_non_blocking removes the implicit dependency. The VRAM assertions named one host; the invariant is total >= 1 GiB. Lambda (RTX 4090): cuda_graph_tests + driver_and_context + stress_and_advanced + cublas_tests 44/44. Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…5c81 inside the disposable-runner image with the host toolkit mounted Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…oint in the workspace — pre-create it as the runner (rebuild 34488955316 died at mkdir after a green 1m54s build) Pmat-Ticket: PMAT-1096 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…rst 'Host layout' step exports CI_TARGETS_ROOT / CI_CARGO_ROOT / SCCACHE_HOST_DIR / CI_REGISTRY / IMAGE with the intel clean-room defaults, 33 hardcoded sites read them; byte-identical on intel (#3100) Also: check_runner_labels.sh accepts the `build` pool label. No job changes where it runs yet (BP-3). Pmat-Ticket: PMAT-1098 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…asset-target-mountpoint

… box carrying `build` (intel clean-room or yoga-eph); gate on any pool box (operator 2026-09-10: requeue to available capacity; #3100) Pmat-Ticket: PMAT-1096 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…-paths

… box carrying `build` (intel clean-room or yoga-eph); gate on any pool box (operator 2026-09-10: requeue to available capacity; #3100) Pmat-Ticket: PMAT-1096 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…red review job held yoga's only pool slot for 15 min while required jobs queued (#3100) Pmat-Ticket: PMAT-1096 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

) replaces the 67-B1 inline asserts; both impl-estimates rows kept; roadmap merged by id

…ver had dropped ids main added

…eck_roadmap_sorted)

noahgift · 2026-09-12T13:24:52Z

Dequeued from the 0.67.0 merge queue, moved to 0.68.0.

Why. This PR's merge-group CI was RED on guard-tree → scripts/check_roadmap_diff_additive.sh (44 checks, 1 failed). The branch edits docs/roadmaps/roadmap.yaml against a base that has since moved: #3136 squash-merged ahead of it, so this branch's roadmap rendering is no longer an additive diff against main. Same failure took #3046 out of the queue an hour earlier. Three groups in a row, one cause.

The remedy is in the guard's own header, and it is mechanical:

git fetch origin main && git rebase origin/main     # or merge
python3 scripts/roadmap_trim.py                     # or: roadmap_diff.py trim
bash scripts/check_roadmap_diff_additive.sh         # must exit 0 before re-arming

roadmap_trim.py rebuilds the file as base's bytes for every entry that is unchanged or merely re-serialised, keeping this branch's rendering only for entries it genuinely adds or edits.

A second finding, worth its own ticket. guard-tree reported this failure at 11:12:00Z. The gate job did not report it until 12:57:26Z — 1 h 45 min later — because gate waits for workspace-test, which was starved behind two other merge-group workspace-test jobs on the same intel box. The queue spent two hours of three-box fleet time to surface a failure that was decided in the first minute. A group whose guard-tree is RED should fail the group immediately.

Not a judgement on the change itself. APR-RELEASE-001 §4: scope is assigned to a train after the fact — 0.67.0 contains whatever merged before its cut. This is on the post-tag re-arm list; nothing else is required of it beyond the trim above.

…t in 0.67.0 Verified every `#NNNN` in the 0.67.0 section against `git log v0.66.0..origin/main` and against each PR's state. Four rows were false: #3068 OPEN — dequeued from the merge queue; guard-tree RED on check_roadmap_diff_additive.sh. Moved to 0.68.0. #3139 OPEN — same guard, same cause. Moved to 0.68.0. #3064 MERGED 2026-09-09T22:20Z — its merge commit is an ANCESTOR OF v0.66.0. It shipped in the previous release; listing it here bills it twice. #3065 MERGED 2026-09-09T18:00Z — likewise an ancestor of v0.66.0. The last two are the more interesting defect: "merged after the last CHANGELOG entry" is not the same predicate as "merged after the last tag", and only the second one is true of a release. `git merge-base --is-ancestor <merge-sha> v0.66.0` answers it and is what was run here. A release note that lists work the release does not contain is a false record, and it is the kind that survives because nobody re-derives it. The scope rule is APR-RELEASE-001 §4: a train contains whatever merged before its cut, by definition -- so the fix is to shrink the list, never to hold the train for it. Pmat-Ticket: PMAT-1098 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…, #3068/#3139/#3175 dequeues, the T-0 correction table, G3.EX GO, and the model return (opus -> fable) recorded Pmat-Ticket: PMAT-1098 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

noahgift · 2026-09-12T18:45:26Z

Dequeued from the merge queue at 18:46Z, and this is not a verdict on the change.

Its group run 34706411340 was RED on exactly one guard:

FAIL  scripts/check_roadmap_diff_additive.sh [run]
      | check_roadmap_diff_additive.sh: merge-base(origin/main, HEAD) is unresolvable (shallow checkout)
      | and HEAD is not a merge commit nor a commit on the origin/main tip

That is #3186 — the stacked merge-group resolver defect — and this PR was sitting at queue position 2, ahead of its own fix, which is #3187 at position 3. Every group built for it there is red by construction and holds a build slot the queue could use, so it was removed under APR-RELEASE-001 §3.4 ("dequeue only a group that is known RED, and fix or trim it rather than park it").

Re-arm it once #3187 is on main: gh pr merge 3068 --repo paiml/aprender --squash --auto. Nothing about the change itself needs to alter.

check_roadmap_diff_additive.sh reported base id(s) missing at head: this branch forked before they landed on main, and a plain update takes one side of the 847-entry file whole. Resolved as main's roadmap plus this branch's own new entries. The other three roadmap guards pass on the result. Pmat-Ticket: PMAT-1098 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* release: 0.67.0 bump-version.sh 0.67.0 (every workspace, facades included; --check green) and the CHANGELOG [0.67.0] section, per docs/specifications/06x-release-schedule.md §4.2. After this merges, rel-067-autopilot runs §4.3-4.8: pre-publish dogfood, tag, release, the sixteen assets by command, cascade, host receipts, close. Pmat-Ticket: PMAT-1098 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(release): 0.67.0 CHANGELOG — paiml#3046 left the merge queue (guard-tree: roadmap diff not additive on its group) and cannot precede the cut; it ships in 0.68 Pmat-Ticket: PMAT-1098 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(release): the 0.67.0 CHANGELOG claimed four changes that are not in 0.67.0 Verified every `#NNNN` in the 0.67.0 section against `git log v0.66.0..origin/main` and against each PR's state. Four rows were false: paiml#3068 OPEN — dequeued from the merge queue; guard-tree RED on check_roadmap_diff_additive.sh. Moved to 0.68.0. paiml#3139 OPEN — same guard, same cause. Moved to 0.68.0. paiml#3064 MERGED 2026-09-09T22:20Z — its merge commit is an ANCESTOR OF v0.66.0. It shipped in the previous release; listing it here bills it twice. paiml#3065 MERGED 2026-09-09T18:00Z — likewise an ancestor of v0.66.0. The last two are the more interesting defect: "merged after the last CHANGELOG entry" is not the same predicate as "merged after the last tag", and only the second one is true of a release. `git merge-base --is-ancestor <merge-sha> v0.66.0` answers it and is what was run here. A release note that lists work the release does not contain is a false record, and it is the kind that survives because nobody re-derives it. The scope rule is APR-RELEASE-001 §4: a train contains whatever merged before its cut, by definition -- so the fix is to shrink the list, never to hold the train for it. Pmat-Ticket: PMAT-1098 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(release): drop the 0.67.0 "### Added" heading left empty by the four false rows A section heading with no rows under it reads as "nothing was added" only if you notice the heading is empty; more often it reads as a rendering bug. Removing the heading says the same thing without the ambiguity. Pmat-Ticket: PMAT-1098 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(release): revert the CHANGELOG heading sweep — it deleted 31 headings across eight years of history The previous commit removed every "### " heading with no "- " bullet under it. Thirty of the thirty-one it matched were historical sections whose content is PROSE, not bullets: "### Migration Guide", "### Quality Metrics", "### TOP 10 Algorithms - Complete List". A heading with no bullets is not an empty heading. This reverts it and removes exactly one heading: the 0.67.0 "### Added", which is followed immediately by the next heading with nothing at all between them. The predicate is now "nothing before the next heading", not "no bullets", and it is scoped to the 0.67.0 section. Pmat-Ticket: PMAT-1098 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(release): the 0.67.0 CHANGELOG missed four PRs merged after the v0.66.0 tag — the CUDA-asset rebuild rows (paiml#3072, paiml#3074, paiml#3086) and the PMAT-1096 receipt (paiml#3085) T-0 rule: CHANGELOG from merged PR titles since the last tag. Predicate: `git log v0.66.0..origin/main` titles minus CHANGELOG rows = 4 → now 0. Each merge commit is NOT an ancestor of v0.66.0 (`git merge-base --is-ancestor` rc=1 for 2c584a1 43b8d8e 58c3dda a4b27f3). Rows appended to ### Changed; narrative gains one sentence on the glibc 2.31 floor. Headings 224 → 224; 0.67.0 rows 27 → 31. release_notes.md carries the same four rows. Pmat-Ticket: PMAT-1098 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

…t check carried a DET002 and a SEC010 that no PR check ran (paiml#3198) The 0.67.0 pre-publish dogfood is NO-GO on bashrs for the third time in one train, each time on a file a PR merged after the previous fix cleared the tree (paiml#3196). This time scripts/cuda_rust_fleet_check.sh from paiml#3068: DET002 on the receipt timestamp and SEC010 on the cd into --repo. Both fixes are mutation-tested on the real file with the dogfood's own invocation (bashrs lint --no-ignore --level error): - NOW derives from SOURCE_DATE_EPOCH with a wall-clock fallback, so a re-run can pin the receipt time; restoring the bare date brings DET002 back. - REPO (from REPO_ROOT or --repo) is refused before first use unless absolute and free of '..'; removing the guard brings SEC010 back. Both refusals are exercised. - --self-test still passes. Gating findings over the whole surface: 2 -> 0. Pmat-Ticket: PMAT-1098 Co-authored-by: Noah Gift <claude@noahgift.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

…e 0.67.0 train three times on findings no PR check saw (paiml#3196) (paiml#3199) dogfood.sh's bashrs row is the only place the SEC/DET/IDEM filter ran, so a PR could add a finding freely and the train paid for it at the pre-publish gate: 17:33Z (paiml#3115, paiml#3127 -> fixed by paiml#3188), 20:58Z (paiml#3187 -> paiml#3194), 22:48Z (paiml#3068 -> paiml#3198). Three fixes round the merge queue while the tag waited. scripts/check_bashrs_gate.sh is that gate lifted out where the finding is cheap: same surface enumeration (git ls-files, N+1 receipt asserted), same positive control (a DET002 sentinel that must fire or the run is ENV/2), same code filter. Its --self-test carries a drift gate that greps dogfood.sh for the identical rule tuples, a classifier case table, a must-RED fixture repo built from the incident's exact construct (a --repo argument parsed into a variable and used in a subshell cd; cd "$1" alone is not flagged, measured), its must-GREEN twin carrying paiml#3198's validation, and a vacuity case (gitignored script -> exit 2, named). Measured on real trees: main @ 0b6f94c -> exit 1, names cuda_rust_fleet_check.sh:129 DET002 and :201 SEC010; paiml#3198's tree -> exit 0. 3 s over 294 files. Wired as a step of guard-cargo, which gate already requires. dogfood.sh is untouched in this change; consolidating its inline row onto this script is the follow-up named in paiml#3196. Pmat-Ticket: PMAT-1098 Co-authored-by: Noah Gift <claude@noahgift.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

noahgift and others added 11 commits September 9, 2026 15:08

roadmap: mint PMAT-1095 — 0.67 CUDA Rust fleet readiness (gx10 / yoga…

a924390

… / lambda-vector) Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J9cSQynVPYeUkQ2i7ccvrs

noahgift added this to the 0.67.0 milestone Sep 9, 2026

noahgift enabled auto-merge September 9, 2026 15:10

noahgift added 2 commits September 9, 2026 17:11

noahgift mentioned this pull request Sep 9, 2026

EPIC: nvidia-cuda-rust-library-integration.md — stabilize GPU quality with NVIDIA's CUDA Rust libraries (0.67) #3062

Open

6 tasks

noahgift mentioned this pull request Sep 10, 2026

EPIC: release train 0.67.0 — 06x-release-schedule.md §3 (0.67.0) #3078

Closed

6 tasks

noahgift and others added 4 commits September 10, 2026 15:14

evidence(cuda-rust-fleet): receipts carry ~/ not /home/noah/ — the sh…

b962678

…ipped-path ratchet was +4 on the merged tree (guard-tree red on run 34368743080) Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

merge origin/main into PMAT-1095-cuda-rust-fleet (roadmap/estimates u…

bb0b32e

…nions) Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

evidence(67-B1): cuda lib suite receipts — gx10 and yoga GREEN at bd0…

6d32b5e

…5c81 inside the disposable-runner image with the host toolkit mounted Pmat-Ticket: PMAT-1095 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

noahgift mentioned this pull request Sep 10, 2026

PP-26 witness RED on gx10 (GB10 sm_121, CUDA 13.3): c=16 batch loses slot invariance at token 31; c=4 and lambda sm_89 stay invariant to 128 #3096

Open

noahgift and others added 7 commits September 10, 2026 16:29

ci(release): the bullseye CUDA build left a root-owned target/ mountp…

d457931

…oint in the workspace — pre-create it as the runner (rebuild 34488955316 died at mkdir after a green 1m54s build) Pmat-Ticket: PMAT-1096 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Merge branch 'PMAT-1098-build-pool-any-of-three' into PMAT-1096-cuda-…

629869c

…asset-target-mountpoint

Merge commit '3b56d009bc256be68c4f1d17fca32795b0b16b9a' into fix-3068…

891d3c5

…-paths

ci(build-pool): the pr-review-* jobs stay on clean-room — a non-requi…

f5732c0

…red review job held yoga's only pool slot for 15 min while required jobs queued (#3100) Pmat-Ticket: PMAT-1096 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

noahgift enabled auto-merge September 12, 2026 09:16

Noah Gift added 2 commits September 12, 2026 11:23

merge origin/main into PMAT-1095 — main's assert_vram_floor (train #3127

21f7697

) replaces the 67-B1 inline asserts; both impl-estimates rows kept; roadmap merged by id

roadmap: main's file plus this PR's own entries — the 3-way-by-id dri…

aa90f98

…ver had dropped ids main added

noahgift mentioned this pull request Sep 12, 2026

workspace-test on ARM64: gx10 runs the quick tier 3–4× faster than intel — route it arch-neutral once the four aarch64-only reds are fixed #3138

Closed

roadmap: PMAT-1095 after its numeric predecessor among PMAT peers (ch…

ff8727f

…eck_roadmap_sorted)

This was referenced Sep 12, 2026

EPIC 0.70: architectural investigation — ggmlc-derived inference runtime ideas (quorum-decided) #3144

Open

release: 0.67.0 #3145

Merged

noahgift added this pull request to the merge queue Sep 12, 2026

noahgift removed this pull request from the merge queue due to a manual request Sep 12, 2026

noahgift modified the milestones: 0.67.0, 0.68.0 Sep 12, 2026

noahgift mentioned this pull request Sep 12, 2026

A merge group whose guard-tree is RED still burns ~2 h of intel workspace-test time before gate reports it #3177

Open

noahgift added this pull request to the merge queue Sep 12, 2026

noahgift mentioned this pull request Sep 12, 2026

resolve_base refuses every STACKED merge-group entry (queue position ≥2): two of every three queue builds are RED by construction #3186

Closed

noahgift removed this pull request from the merge queue due to a manual request Sep 12, 2026

noahgift added this pull request to the merge queue Sep 12, 2026

noahgift mentioned this pull request Sep 12, 2026

Every open PR is roadmap-non-additive, so nothing can enter the merge queue and all three boxes sit near 10% — 31 of 31 measured #3195

Closed

Merged via the queue into main with commit f7727b4 Sep 12, 2026
1 check failed

noahgift deleted the PMAT-1095-cuda-rust-fleet branch September 12, 2026 22:20

Navigation Menu

Uh oh!

feat(PMAT-1095): 0.67 CUDA Rust fleet readiness — fix the two wrong-host GPU tests, add scripts/cuda_rust_fleet_check.sh with per-host receipts (gx10 PASS, yoga PASS, lambda-vector blocked on a reboot) - #3068

feat(PMAT-1095): 0.67 CUDA Rust fleet readiness — fix the two wrong-host GPU tests, add scripts/cuda_rust_fleet_check.sh with per-host receipts (gx10 PASS, yoga PASS, lambda-vector blocked on a reboot)#3068
noahgift merged 39 commits into
mainfrom
PMAT-1095-cuda-rust-fleet

noahgift commented Sep 9, 2026

Uh oh!

github-actions Bot commented Sep 9, 2026 •

edited

Loading

Uh oh!

Uh oh!

noahgift commented Sep 12, 2026

Uh oh!

Uh oh!

noahgift commented Sep 12, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

noahgift commented Sep 9, 2026

What it does

Receipts (committed, evidence/cuda-rust-fleet/)

What went wrong on the way, owned

Gates

Uh oh!

github-actions Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

noahgift commented Sep 12, 2026

Uh oh!

Uh oh!

noahgift commented Sep 12, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Receipts (committed, `evidence/cuda-rust-fleet/`)

github-actions Bot commented Sep 9, 2026 •

edited

Loading