ci(silicon): the two pending axes had runners — promote them, and give each one a job

[noahgift]

silicon-nightly is not broken — it is the anti-rot rule firing

Four runs red (09-09, 09-10, 09-11, 09-13) and two cancelled. The coverage job's verdict, from run 34747695348:

  ok        x86_64-cpu           served by intel-clean-room-10
  ok        aarch64-cuda-sm121   served by gx10-blackwell
  PROMOTE   x86_64-cuda-sm89     marked pending:361, but yoga-gpu can serve it NOW
  deferred  aarch64-cuda-sm87    pending:361 — no runner yet
  PROMOTE   aarch64-macos        marked pending:361, but mini-m4 can serve it NOW

FAIL: silicon coverage does not match the policy.

.github/silicon-coverage.txt says a pending axis cannot rot: "the preflight fails if a runner able to serve the selector APPEARS, because at that point the only thing standing between us and the coverage is this line." Both tripped it. The 08:26 run saw only yoga; mini-m4 registered later the same day, which is why the live guard now reports two.

I first assumed the matcher was case-sensitive (mini's labels are ARM64/macOS, the selector is arm64/macos). It is not — labels_contain lowercases both sides and a committed self-test row pins exactly that. Running the guard against the live fleet is what settled it.

Why promoting only the lines would have been a lie

required reads as "this architecture is tested." Neither axis was being tested by this lane, and for sm_89 the nearest thing is conditional: cuda-unit reaches yoga only on a pull_request and only when gpu_touched == 1. Conditional coverage is not an axis. So each promoted axis gets a job running the same arch-sensitive subset the other two run, plus summary.needs, the reported table, and the rc loop that turns a non-success axis into a lane failure.

axis	runner	why it was pending	why it isn't now
x86_64-cuda-sm89	yoga-gpu	the x86_64 GPU leg ran on lambda-labs, which must never be a CI host (paiml/infra#359), so #2740 made cuda-nightly gx10-only	yoga-gpu is a self-hosted Ada card — the reason is gone
aarch64-macos	mini-m4	no runner	the only darwin/arm64 silicon the repo has

Everything else in this lane is Linux, so macOS assumptions are invisible to it by construction. The first full darwin run of the workspace found two — a /proc/meminfo read, and an ETXTBSY premise that is a Linux guarantee and simply is not true on XNU.

Release contention

yoga-gpu is also binary-release.yml's x86_64 CUDA asset runner, so the new job takes the treatment gx10 got in PP-19 — a named group, so a nightly and a release cut queue behind each other instead of contending:

ok    silicon-nightly.yml:aarch64-cuda-sm121 (gpu runs-on) group=perf-gx10 cancel-in-progress=false
ok    silicon-nightly.yml:x86_64-cuda-sm89   (gpu runs-on) group=perf-yoga cancel-in-progress=false
49 job(s) scanned, 8 perf-sensitive, 0 finding(s)

Measured after the change

$ bash scripts/check_silicon_coverage.sh
  ok        x86_64-cpu           served by intel-clean-room-10
  ok        aarch64-cuda-sm121   served by gx10-blackwell
  ok        x86_64-cuda-sm89     served by yoga-gpu
  ok        aarch64-macos        served by mini-m4
  deferred  aarch64-cuda-sm87    pending:361 — no runner yet

axes declared: 5  (required 4: covered 4, MISSING 0; deferred 1: PROMOTABLE 0)
OK: every required axis has a runner, and every deferred axis is still blocked.

check_silicon_packages.sh and check_perf_concurrency_groups.sh both pass.

What I have not measured: neither new job has ever run. The darwin one is close to work that has run (mini's macos-arm64 leg in #3205 tests aprender-compute natively and passes), but --release on a 16 GB box and the yoga leg are both unmeasured — which is why both carry a 90-minute timeout and CARGO_BUILD_JOBS: 6 on mini rather than a tight bound guessed from nothing. If either goes red on its first night, that is the andon, not a regression.

no-close: silicon-nightly's red is the policy working; this promotes the two axes whose blockers cleared and leaves aarch64-cuda-sm87 deferred, which keeps the lane's remaining gap a value rather than an absence.

🤖 Generated with Claude Code

[github-actions]

§13.11 rung 1 — quorum shadow verdict

S13-SHADOW pr=3230 head=3795a3284633920bc87fd219471c139daf4c280e verdict=REFUSE class=Q1 arm_rc=1

Shadow mode: this records a verdict and merges nothing. A refusal
to arm is not a block (§13 adds zero rows to §7) — the pull request is
exactly as green as it was.

[noahgift]

Closing: this duplicates #3060, which is already in the merge queue at position 8 and solves the same problem more deeply. I wrote it without checking which open PRs already owned .github/silicon-coverage.txt, which is my error — the two would have conflicted on the file header.

#3060 rewrites the policy's semantics from the outside in:

-#   required        this axis MUST have a runner that can serve its selector.
+#   required        this axis MUST have RUN inside its cadence window.
...
+# COVERAGE IS A RUN, NOT A RUNNER (R-5, YOGA-NIGHTLY-001 §9.2).

That is the same objection I raised in this PR's own description — "required reads as 'this architecture is tested'" — and #3060 answers it properly, with a job:<name glob> column and a cadence window, instead of my weaker "promote the line and add a job so the claim is not empty". It also carries the sm_89 promotion with the infra citation this PR lacked (yoga as a sanctioned org-scoped runner in the gpu-x86 group, paiml/infra docs/specifications/yoga-nightly-job.md, YOGA-NIGHTLY-001 R-4).

What is not in #3060 and still needs doing: the aarch64-macos axis. mini-m4 registered today, and the live guard now reports it promotable:

  PROMOTE   x86_64-cuda-sm89     marked pending:361, but yoga-gpu can serve it NOW
  PROMOTE   aarch64-macos        marked pending:361, but mini-m4 can serve it NOW

so silicon-nightly stays red on the macos half even after #3060 lands. I'll re-do that piece on top of #3060's semantics — as a run inside a cadence window with a job: binding, not as a runner-exists claim — rather than stack it CI-dark under a queued PR.

Two things worth keeping from the work here, both already filed elsewhere:

• the reason silicon-nightly has been red is the policy's own anti-rot rule firing, not a broken lane;
• mini-m4 sits at 0.0% occupancy over both the 1h and 4h windows while being the only darwin/arm64 silicon the repo has. That is ci(mini): macos-arm64 leg on mini-m4 (NEON tests + core check) and the darwin nightly comes back #3205's job to fix.

🤖 Generated with Claude Code