diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 9ee1e3161a..02e9952683 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1979,374 +1979,42 @@ jobs: shell: bash run: bash scripts/mutate_vendored_schemas_guard.sh - # ── PRREV-006: the receipt guard, wired ────────────────────────────────── + # ── PRREV-006's receipt job: DELETED 2026-09-08 (BSE-15, BSE-001 §4 wave 5) ── # - # PR-REVIEW-SKILL-002 v2 §9 row 6 is one sentence, and the `if:` below is what - # it is about: + # `pr-review-receipt` ran two mutation sweeps on a clean-room runner on every + # push to every open PR — 150 minutes budgeted, and the single largest consumer + # of fleet runner-hours in the 30 days to 2026-09-07 (scripts/ci_job_minutes.sh). + # It is gone rather than made dispatch-only, because the query says nothing was + # reading it and a dispatch-only job that cannot run is worse than no job: # - # job-level `if:`, NOT a workflow-level `paths:` filter — a path-filtered - # required check never reports and blocks branch protection forever. + # $ grep -rn 'needs:' .github/workflows/*.yml | grep pr-review-receipt # -> no match + # $ grep -rn 'download-artifact' .github/workflows/*.yml # -> nightly.yml:141 only, + # # its own build assets + # $ gh api repos/paiml/aprender/branches/main/protection --jq ... # -> ["ci / gate", "workspace-test"] # - # A `paths:` filter stops the WORKFLOW from running at all on a PR that misses - # it, so a check the ruleset requires never produces a check run: the PR sits - # PENDING and nothing merges. This file carries BOTH required checks, so that - # is a deadlock rather than a hypothesis. A job-level `if:` is the opposite - # shape — the workflow runs, the job reports `skipped`, and `gate` reads the - # result. `mutants` below is the in-repo precedent. + # The job also uploaded no artifact, so there was nothing for a consumer to read. + # `gate` had already stopped reading it (PP-066 C0-5, PRQ-013, #2982: a + # head-defined receipt job is a check the PR under review can edit). # - # THE CONDITION IS A PRECONDITION, NOT A DODGE. The subject of this job is the - # receipt for ONE pull request, at evidence/pr-review///. On - # push, merge_group and workflow_dispatch there is no , so there is nothing - # for the check to be about. Both polarities of that claim are checked, not - # asserted: check_pr_review_wiring.sh (guard-tree, above) evaluates - # this exact expression against all four events and fails if it runs where - # there is no PR or skips where there is one. + # WHAT STILL RUNS, and where — this is not a hole, it is a move: + # * check_pr_review_receipt.sh, check_pr_review_counts.sh and + # check_pr_review_arm4.sh are `scripts/check_*.sh`, so guard_tree.sh's + # derived universe (`git ls-files 'scripts/check_*.sh'`, BSE-001 PR-A) runs + # them in `guard-tree`, which `gate` needs. Adding a guard never edits this + # file, and removing this job does not un-wire them. + # * the PR's own receipt is judged from the BASE by pr-review-quorum.yml + # (`pull_request_target` + `merge_group`), which invokes + # check_pr_review_arm4.sh itself and consumed nothing from this job. # - # FIVE ARMS. Arm 0 is text-only and runs first; the first two after it are not - # the same check twice: - # 0. the counts these files state as MEASURED, against what the tree derives. - # The skill reviewed its own PR and found five stale ones — "119/119" in - # four places over a set of 185, "65 tests" over 121, "22-row" over 26 — - # each true when written and stale by the commit that shipped it. Two of - # them are in the step NAMES below. check_pr_review_counts.sh derives the - # three that a tree can derive and fails on any file that disagrees; its - # --self-test writes each of the three shipped defects back in and requires - # RED, then grows the tree three ways and requires RED again. - # 1..4 as before: - # 1. the guard over one GREEN and one RED fixture. It is the only arm that - # survives an unobtainable bats, and it is the literal invocation - # check_guards_are_wired.sh scans for — a guard named in a comment is not - # wired. GREEN and RED together, because "refuse every receipt" reads - # green against RED fixtures alone. - # 2. the 43-row fixture table (§6.3's fourteen, the twelve PRREV-008 and - # PRREV-009 added, S3.E's nine, and PRREV-020's two) under bats, positive - # controls included. - # 3. the guard's own mutation set (§6.4). §8 fixes guard_mutation_score at - # 100% with "no ratchet, it is a one". Wiring a guard and leaving its - # falsifier dark would make the thing every other verdict rests on unable - # to be SHOWN to fail — the epic's own most common defect class (§11). - # 4. this PR's own receipt, which arms itself (below). - # - # NOT IN `gate`'s needs, for vendored-schemas' reason plus a sharper one: - # nothing produces a receipt until PRREV-005, so a blocking receipt check today - # would red every PR including the one that lands PRREV-005 — a gate whose own - # remedy it makes impossible, which is how `apr test llm` shipped permanently - # unreachable. §9 step 7 makes the backtest the acceptance test "before it is - # enabled", so promotion is PRREV-007's. Arm 4 re-evaluates the arming - # condition on EVERY run rather than leaving it to a comment: the comment that - # said to promote check_hardcoded_paths.sh "once the fleet carries pmat >= - # 3.32.0" is two steps above this one, and 20 machine-specific paths landed - # through the gap while nothing re-read it. - pr-review-receipt: - # X64 for now, NOT arch-neutral (#3100): the first completed run on gx10 (train #3127 run - # 34648948737, gx10-pool1) left mutant reject-76-drop (check_pr_review_receipt.sh L1236, - # the tool.driver.name jq filter) SURVIVING — 232/233 killed — while every x86 run kills - # it. A job class goes to a box only once it is proven there; the arm64 survivor is - # its own ticket, and this line moves back when it is killed on gx10. - runs-on: [self-hosted, X64, Linux, clean-room] - # 150, not 120: Arm 3 alone measured 3091s (51.5 min) on an idle 48-core box over - # the 185-mutant set, and PRREV-015 adds Arms 5 and 6 — an 83-row bats table and a - # second, 134-mutant sweep over scripts/pr_review_quorum_arm.sh. This number has - # already been wrong once here for exactly this reason (60 against a measured 51.5 - # left 8 minutes for four other arms, which is not headroom; it is a job that times - # out the first time the runner is busy, and a cancelled job reads as a failure - # nobody can distinguish from a real one). - timeout-minutes: 150 - if: github.event_name == 'pull_request' - env: - # The `runner` context is NOT available in a job-level `env:` block — - # GitHub rejects the workflow with `Unrecognized named-value: runner` - # (#2791). $RUNNER_TEMP is read inside the run blocks instead. - PR_NUMBER: ${{ github.event.pull_request.number }} - PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }} - steps: - - name: Checkout - uses: actions/checkout@v7 - - # The guard REFUSES to run without minisign, check-jsonschema, bats and jq: - # "a gate that cannot run must not read green". They go into a per-JOB - # private root under $RUNNER_TEMP, never ~/.local/bin: mac-server hosts 16 - # runners under one $HOME, and a concurrent writer to a shared bin - # directory is what made Coverage Nightly report an EMPTY coverage figure - # (#2353). $RUNNER_TEMP and not /tmp, because /tmp half-persists on a - # long-lived box and is emptied at boot (#2791). Versions and digests are - # pinned in the script, which exits 2 — distinct from the guard's own 1 — - # when the BOX cannot answer, so a broken runner is never read as a broken - # tree. - - name: Install the tools the receipt guard refuses to run without - shell: bash - env: - GITHUB_TOKEN: ${{ github.token }} - run: | - set -euo pipefail - bash scripts/install_pr_review_tools.sh "$RUNNER_TEMP/pr-review-tools/bin" - echo "$RUNNER_TEMP/pr-review-tools/bin" >> "$GITHUB_PATH" - - # Arm 0. Text-only, no build, and FIRST: the arms below quote numbers in - # their own step names, and a step named after a stale number is how "65 - # tests" and "22-row" survived four commits of growth. Self-test before - # check, because a guard whose RED has never been seen is a guard whose - # GREEN is a count of files (Verification Discipline #7). - - name: "Stated-count guard: every stated-count drift turns it RED, both polarities" - shell: bash - run: bash scripts/check_pr_review_counts.sh --self-test - - name: The counts these files state as measured equal the derived ones - shell: bash - run: bash scripts/check_pr_review_counts.sh - - # Arm 1. The receipts name commits in a purpose-built repository, not in - # aprender: for any commit reachable from aprender's own origin/main, - # `git merge-base origin/main X` is X itself, so the base_sha row would - # pass vacuously. Both exit codes are read from the guard itself, never - # through a pipe. - - name: The guard accepts an honest receipt and rejects a self-review - shell: bash - run: | - set -euo pipefail - FIX=tests/fixtures/pr-review - "$FIX/make-fixture-repo.sh" "$RUNNER_TEMP/pr-review-fixture-repo" > /dev/null - export PR_REVIEW_REPO="$RUNNER_TEMP/pr-review-fixture-repo" - export PR_REVIEW_PUBKEY="$FIX/keys/pr-review-test.pub" - - bash scripts/check_pr_review_receipt.sh "$FIX/row-14-complete-gpu-review" - echo "GREEN discrimination row accepted, as it must be" - - rc=0 - bash scripts/check_pr_review_receipt.sh "$FIX/row-08-self-review" || rc=$? - if [ "$rc" -ne 1 ]; then - echo "row 08 (reviewer_actor == author_actor) exited $rc, expected 1." >&2 - echo "A guard that accepts a self-review has no verdict to give (§5, §7)." >&2 - exit 1 - fi - echo "RED discrimination row rejected under B2, as it must be" - - # Arm 2. Every row of §6.3, plus the positive controls, plus the - # must-match/must-not-match tables behind the three trigger regexes. This - # file is 165 tests and was reachable from NO workflow until this job — the - # dark-target class this repository has hit with benches and with 58 cgp - # tests. Both numbers here are recomputed by check_pr_review_counts.sh; the - # commit that wired this step wrote 65 and 22 over a file that already held - # 165 tests and 37 rows. - - name: The 43-row fixture table and both polarities of every trigger regex - shell: bash - run: bats tests/pr-review.bats - - # Arm 3. §6.4: mechanically flip each validation branch and drop each - # required-field check, target 100% kill. - # - # The script's DEFAULT parallelism, because that is the only setting whose - # wall time has been measured: 185/185 killed in 3091s (51.5 min) at - # --jobs 12 on a 48-core box, 2026-08-30. - # - # THE SET IS 233/233 TODAY — PRREV-015 added S3.E, PRREV-020 its availability - # rule and PRREV-019 the survivors-shape rule, one `reject B` site each and - # so two mutants each. So that wall time is a - # LOWER BOUND and not a current measurement. It is left as the number that - # was actually measured, with its date and its set size beside it, rather - # than rescaled: a scaled estimate wearing the shape of a measurement is - # the defect this whole job exists to catch. Re-measure before quoting it. - # The set is DERIVED — one `drop` and one - # `flip` per `reject B` site, rescanned from the guard on every run — so - # its size moves when the guard does; `scripts/mutate-guard.sh --list` - # prints it, and check_pr_review_counts.sh fails this file when the number - # written here is not the number the script derives. An earlier draft of - # this step wrote --jobs 8 with a plausible-sounding reason and no - # measurement behind it, which is how a number becomes folklore. - # - # THE TIMEOUT WAS A THIRD DEFECT IN THIS JOB, and the same one: this - # paragraph said "timeout-minutes on this job is 60 against that 19 - # minutes; a slower runner has headroom". 19 minutes was 1125s over the - # 119-mutant set. The set is 185 and the same box takes 3091s — 51.5 - # minutes against a 60-minute cap, on an IDLE box, before Arms 0-2 and 4 - # have run at all. Headroom of 8 minutes is not headroom; it is a job that - # times out the first time the runner is busy, and a cancelled job reads - # as a failure nobody can distinguish from a real one (#2792's cargo-audit - # eviction, and "N runs cancelled at the SAME elapsed time is a timeout, - # not a flake"). Raised to 120. - # - # AND A FOURTH: THIS STEP HAD NO `if:` AT ALL, so the full sweep ran on every - # pull request including docs-only ones -- while §3.D's own trigger table says - # the guard mutation set is required when the diff "touches scripts/check_*.sh, - # dogfood.sh, ci.yml gate logic, or a contracts/*.yaml falsifier" and that - # docs / non-code is "not triggered". CI contradicted the specification it - # enforces, and it was not a tidiness point: it is the queue. Measured - # 2026-09-01 -- three receipt jobs started within nine minutes of each other, - # the runner host's load average went 48 -> 133, and #2836's job was CANCELLED - # at exactly its 150-minute cap having proved nothing. A cancelled job reads as - # a failure nobody can distinguish from a real one. - # - # The decision is a SCRIPT with an 11-row case table, not an inline expression: - # this job's own Arm 4 post-mortem is that inline YAML is reachable from no test - # and its exit-0 branch passed every run it ever had. The table carries both - # polarities -- all seven sweep inputs must answer `yes`, and a docs-only and an - # unrelated-source diff must answer `no`, without which "always yes" reads green - # -- plus a fail-closed row and a row that fails when build_tree copies a path - # the list does not name. - - name: "Does this diff change anything the sweep reads? (§3.D trigger)" - id: sweep_needed - shell: bash - run: | - bash scripts/pr_review_sweep_needed.sh --self-test - BASE=$(git merge-base origin/main "${PR_HEAD_SHA}" 2>/dev/null) || BASE="" - if [ -z "$BASE" ]; then - echo "needed=yes" >> "$GITHUB_OUTPUT" - echo "::notice::merge-base unavailable; running the sweep (fail-closed)" - exit 0 - fi - # FAIL-CLOSED. An earlier draft assigned OUT and appended it: if the script - # crashed, OUT was empty, `needed` was never set, and `needed == 'yes'` is - # false -- so a CRASHING TRIGGER SILENTLY SKIPPED THE SWEEP. That is the - # fail-open shape this repository exists to remove, in the guard that gates - # the guard. Status is read from the command, never through a pipe, and any - # outcome that is not a well-formed `needed=no` runs the sweep. - set +e - OUT=$(bash scripts/pr_review_sweep_needed.sh "$BASE" "${PR_HEAD_SHA}") - rc=$? - set -e - if [ "$rc" -ne 0 ] || [ -z "$OUT" ]; then - echo "needed=yes" >> "$GITHUB_OUTPUT" - echo "::notice::sweep trigger exited $rc with output '$OUT'; running the sweep (fail-closed)" - exit 0 - fi - case "$OUT" in - needed=no) echo "needed=no" >> "$GITHUB_OUTPUT" ;; - needed=yes) echo "needed=yes" >> "$GITHUB_OUTPUT" ;; - *) echo "needed=yes" >> "$GITHUB_OUTPUT" - echo "::notice::sweep trigger printed an unrecognised answer '$OUT'; running the sweep (fail-closed)" ;; - esac - echo "::notice::sweep trigger -> $OUT" - - - name: The guard's mutation set must kill 100% (§6.4, §8 fixes this at one) - if: steps.sweep_needed.outputs.needed == 'yes' - shell: bash - run: bash scripts/mutate-guard.sh - - # A SKIPPED SWEEP IS RECORDED, NEVER SILENT. §3.0's rule one level up: "not - # triggered" and "did not run" must not be the same artifact. - - name: "Sweep not triggered — record why (§3.0)" - if: steps.sweep_needed.outputs.needed != 'yes' - shell: bash - run: | - echo "guard_mutation_score: NOT-TRIGGERED — no sweep input changed in this diff." - echo "Inputs are the six paths build_tree copies plus scripts/mutate-guard.sh;" - echo "see scripts/pr_review_sweep_needed.sh. The score is unchanged from main's." - - # Arm 4. ARMED (PRREV-013). It could previously neither fail nor pass. - # - # The step this replaces was six lines of inline YAML, and it carried BOTH - # halves of the same defect: - # - # * it began `if [ ! -f .github/pr-review.pub ]; then ... exit 0; fi`, - # the key was absent, nothing owned shipping it (SKILL.md said - # PRREV-006, the two comments above said PRREV-005, keys/README.md - # hedged "PRREV-005/006"), and `PR_REVIEW_SIGNING_KEY` appeared in - # zero workflows. So it EXITED 0 ON EVERY RUN IT EVER HAD. §8 fixes - # `receipt_presence` at 100% "no ratchet, it is a one", and the job - # written to enforce that could not fail — a gate that cannot fail, - # inside the job built to prevent gates that cannot fail (§11). - # - # * its armed branch then looked for `evidence/pr-review//`, - # which NO pull request can produce: committing the receipt changes the - # tip, so a directory named after the tip cannot hold the review of the - # tip. A gate that cannot pass, shipped in the same six lines as a gate - # that cannot fail. - # - # Both are fixed in scripts/check_pr_review_arm4.sh, which is a SCRIPT and - # not inline YAML for the reason the `exit 0` survived unnoticed: Arms 1-3 - # are all scripts with case tables, Arm 4 was the one reachable from no - # test, and Arm 4 is the one that broke. Its `--self-test` drives a case - # table, both polarities, including the absent-key row that used to read - # green — deliberately WITHOUT a row count written here, because a count of - # a table that grows is the drift Arm 0 exists to catch. The table is - # hermetic: it runs against tests/fixtures/pr-review/make-fixture-repo.sh, - # not against this repository's history, so a squash-merge cannot expire it. - # The self-test runs FIRST (Verification Discipline #7). - # - # .github/pr-review.pub is now COMMITTED — PRREV-013 generated the keypair, - # committed the public half, re-signed this PR's receipt with the secret - # half, and put that half in the repository secret PR_REVIEW_SIGNING_KEY_B64 - # (base64 of the minisign secret-key FILE; §4.3's `minisign -S -s` takes a - # path, so a signer materialises it before use). The secret half is not in - # this repository and never will be. Rotation is `minisign -G -W`, replace - # .github/pr-review.pub, re-set the secret, re-sign. - # - # A MISSING RECEIPT IS NOW RED (§6.3, verbatim: "a missing receipt is RED, - # not skipped"). This job is deliberately still NOT in `gate`'s needs, so a - # red here does not block a merge; it makes `receipt_presence` a number - # somebody can read instead of a metric nothing measures. Promoting it into - # `gate` is a separate decision and wants 30 samples behind it, per §8. - # - # The step deepens the checkout first: base_sha resolves through - # `git merge-base origin/main `, which a fetch-depth-1 checkout cannot - # answer. The script exits 2 rather than 0 or 1 when the BOX cannot answer, - # so a shallow clone is never read as a clean tree. - - name: "Arm 4 case table: 10 rows, both polarities of the cutoff — the absent-key branch that used to exit 0 forever" - shell: bash - run: bash scripts/check_pr_review_arm4.sh --self-test - # This PR's own receipt (Arm 4 on the live PR) is NOT judged here any more. - # This job runs `on: pull_request`, whose definition GitHub reads from the PR - # head, so a PR could edit the check that judged it (PRQ-013, PP-066 C0-5, - # #2982). It is judged from the BASE in .github/workflows/pr-review-quorum.yml - # (`pull_request_target` + `merge_group`); scripts/check_receipt_gate_base_owned.sh - # refuses a second invocation of check_pr_review_arm4.sh anywhere else. The - # case table above stays: it tests THIS PR's changes to the script, which is - # what a head-defined job is for. - - # Arm 5. S13's refusal table. SHADOW ONLY — scripts/pr_review_quorum_arm.sh is - # invoked here with --explain against committed fixtures and a fixture - # repository, so it can merge nothing, and it is reachable from no step that - # can. S13.11 rung 0. - # - # The bats file exports a RECORDING STUB for gh before any row runs, and that is - # a safety device before it is an assertion: Arm 6 runs this same file against a - # MUTATED arm script, and one of the mutants disables the --explain guard. With - # a real gh on PATH that mutant would try to arm auto-merge on a live pull - # request. The stub records the call and exits 1, and two rows assert the call - # log is empty. - - name: "S13 quorum refusal table: 83 rows, one per refusal path, four that PERMIT" - shell: bash - run: bats tests/pr-review-quorum.bats - - # Arm 6. S13.10 fixes quorum_arm_mutation_score at one with no ratchet, for the - # reason S6.4 gives one tier down and with more force: a surviving mutant is a - # refusal path the arm script STATES and nothing TESTS, and under S13 that is a - # merge nobody authorised. - # - # The set is DERIVED — one `drop` and one `flip` per `refuse Q` site, rescanned - # on every run — so it cannot fall behind the script; check_pr_review_counts.sh - # fails this file when the number written here is not the number the script - # derives. MEASURED at the default --jobs 12 on this 48-core box, 2026-08-31: - # 134/134 killed. It is a SEPARATE script from mutate-guard.sh rather than an - # extension of it, because merging a 15-minute sweep into a 51-minute one buys - # nothing and serialises them. - # Pareto (PMAT-1105 §6): this sweep is ~2 h of the receipt job on EVERY PR and sits in - # the merge queue's ALLGREEN set, so it was the critical path of every merge. Its - # inputs are five paths; a diff that touches none of them cannot change the score, - # which is main's. FAIL-CLOSED exactly like the §3.D trigger above: no merge-base, - # a crashed diff, or anything but a well-formed "no" runs the sweep. - - name: "Does this diff change anything the arm sweep reads? (S13.10 trigger)" - id: arm_needed - shell: bash - run: | - set -euo pipefail - BASE=$(git merge-base origin/main "${PR_HEAD_SHA}" 2>/dev/null) || BASE="" - if [ -z "$BASE" ]; then echo "needed=yes" >> "$GITHUB_OUTPUT"; echo "::notice::merge-base unavailable; running the arm sweep (fail-closed)"; exit 0; fi - set +e - CHANGED=$(git diff --name-only "$BASE" "${PR_HEAD_SHA}" -- scripts/pr_review_quorum_arm.sh scripts/check_pr_review_receipt.sh scripts/mutate_quorum_arm.sh tests/pr-review-quorum.bats tests/fixtures/pr-review/) - rc=$? - set -e - if [ "$rc" -ne 0 ]; then echo "needed=yes" >> "$GITHUB_OUTPUT"; echo "::notice::git diff failed (rc=$rc); running the arm sweep (fail-closed)"; exit 0; fi - if [ -n "$CHANGED" ]; then printf 'arm inputs changed:\n%s\n' "$CHANGED"; echo "needed=yes" >> "$GITHUB_OUTPUT"; else echo "no arm input changed between $BASE and ${PR_HEAD_SHA}"; echo "needed=no" >> "$GITHUB_OUTPUT"; fi - - name: "The arm script's 134-mutant set must kill 100% (S13.10, fixed at one)" - if: steps.arm_needed.outputs.needed == 'yes' - shell: bash - run: bash scripts/mutate_quorum_arm.sh - - name: "Arm sweep not triggered — record why (S13.10)" - if: steps.arm_needed.outputs.needed != 'yes' - shell: bash - run: | - echo "quorum_arm_mutation_score: NOT-TRIGGERED — none of scripts/pr_review_quorum_arm.sh, scripts/check_pr_review_receipt.sh, scripts/mutate_quorum_arm.sh, tests/pr-review-quorum.bats, tests/fixtures/pr-review/ changed in this diff." - echo "The score is unchanged from main's (S13.10); the sweep still runs on every push to main." + # WHAT NO LONGER RUNS IN CI, stated rather than discovered later: the two + # mutation sweeps `scripts/mutate-guard.sh` (Arm 3) and + # `scripts/mutate_quorum_arm.sh` (Arm 6), and the 43-row bats fixture table + # (Arm 2). No other workflow invokes them (`grep -rln` over .github/workflows + # returns nothing), and they are not `check_*.sh`, so guard_tree does not reach + # them. They remain runnable by hand and are the natural content of a nightly. + # Putting the job back on `pull_request` is a RED check, not a quiet reversion: + # scripts/check_pr_review_wiring.sh R1 refuses any ci.yml job that invokes the + # receipt guard, and its case table carries the old `if:` verbatim, asserted FAIL. # --------------------------------------------------------------------------- # PR-REVIEW-SKILL-002 v2 §13.11 RUNG 1 — the shadow lane. diff --git a/docs/roadmaps/roadmap.yaml b/docs/roadmaps/roadmap.yaml index a6fd6383f0..9b9b456c20 100644 --- a/docs/roadmaps/roadmap.yaml +++ b/docs/roadmaps/roadmap.yaml @@ -16703,6 +16703,31 @@ roadmap: estimated_effort: null labels: [] notes: null +- id: PMAT-1078 + github_issue: 3046 + item_type: task + title: 'pr-review-receipt is DELETED: 326.8 h of PR runner-time (floor, measured) for a job nothing needs, reads or requires' + status: inprogress + priority: medium + assigned_to: null + created: 2026-09-08T07:28:32Z + updated: 2026-09-08T11:20:00Z + spec: null + acceptance_criteria: + - 'RESCOPED 2026-09-08 by operator ruling, after a 3/3 FAIL quorum on the dispatch-only attempt: the job is DELETED outright, not moved to workflow_dispatch. The dispatch-only diff gave PR_HEAD_SHA a `|| github.sha` fallback and left PR_NUMBER bound to the pull_request context, which is empty on a dispatch, while check_pr_review_arm4.sh:372 refuses on empty - the job would have been reachable and non-functional, which is worse than absent.' + - 'DONE-IF the deletion is gated on a recorded query: no `needs:` names the job (grep over .github/workflows -> no match); it uploads no artifact and the only download-artifact in the tree is nightly.yml:141 for its own build assets; it is not a required context (branch protection -> ["ci / gate", "workspace-test"]).' + - 'check_pr_review_wiring.sh INVERTS rather than dying with its subject: R4 (the if: polarity table) and its evaluator are deleted with the job; R1 becomes the standing falsifier - no ci.yml job may invoke the receipt guard, with the old `if: github.event_name == ''pull_request''` wiring carried VERBATIM in the case table and asserted FAIL, and a dispatch-only job refused too; R3 is new - the guard must stay a tracked scripts/check_*.sh or R1 holds vacuously. bash scripts/check_pr_review_wiring.sh --self-test -> 10/10, both polarities on every rule.' + - 'scripts/ci_job_minutes.sh makes the figure re-derivable: per-JOB (completed_at - started_at) from the jobs API, started_at not created_at so queue wait is excluded, skipped jobs contribute nothing, and a run count equal to --limit is exit 2 TRUNCATED rather than a silent floor. bash scripts/ci_job_minutes.sh --self-test -> 6/6. Measured 2026-08-08..2026-09-07 on pull_request: pr-review-receipt 326.8 h of a 850.6 h total (a floor; the 96 h this ticket opened with understated it by 3.4x).' + - 'STATED LOSS: the two mutation sweeps (scripts/mutate-guard.sh, scripts/mutate_quorum_arm.sh) and the 43-row bats fixture table no longer run in CI - no other workflow invokes them and they are not check_*.sh, so guard_tree.sh does not reach them. check_pr_review_receipt.sh, check_pr_review_counts.sh and check_pr_review_arm4.sh DO still run, through guard_tree.sh''s derived universe; the PR''s own receipt is judged from the base by pr-review-quorum.yml.' + - 'roadmap.yaml carries ONE addition (this entry) and no other change: check_roadmap_diff_additive.sh -> added=1 lifecycle=0 reserialised=0 deleted=0.' + phases: [] + subtasks: [] + estimated_effort: null + labels: + - kind:code + - pp-066 + notes: 'BSE-15, BSE-001 §4 wave 5. Rescoped from dispatch-only to deletion; the ticket text is the standard the quorum judges the diff against, so it moved with the policy rather than leaving the diff contradicting it.' + - id: PMAT-1079 github_issue: 3045 item_type: task diff --git a/scripts/check_pr_review_wiring.sh b/scripts/check_pr_review_wiring.sh old mode 100755 new mode 100644 index 393d2eee66..6d002107e4 --- a/scripts/check_pr_review_wiring.sh +++ b/scripts/check_pr_review_wiring.sh @@ -1,63 +1,57 @@ #!/usr/bin/env bash -# check_pr_review_wiring.sh — how the PR-review receipt guard is allowed to be -# wired into ci.yml, made mechanical. +# check_pr_review_wiring.sh — the PR-review receipt guard family is reachable, +# and the 150-minute receipt JOB does not come back onto pull requests. # -# WHY THIS EXISTS -# --------------- -# PR-REVIEW-SKILL-002 v2 §9 row 6 states the rule PRREV-006 must not get wrong: +# WHAT CHANGED, AND WHY THIS FILE INVERTED (2026-09-08, BSE-15, BSE-001 §4 wave 5) +# ------------------------------------------------------------------------------- +# Until today this guard asserted that ci.yml carried a `pr-review-receipt` job +# with a job-level `if:`, and R4 asserted that `if:`'s polarity per event. The +# job itself is now DELETED, so R1's old subject ("the job that invokes the +# receipt guard") no longer exists and R3/R4 have nothing to evaluate. Asserting +# the presence of a deleted job is not a weaker guard, it is a broken one. # -# "wire into ci.yml beside existing guards | job-level `if:`, **not** -# workflow-level `paths:` — a path-filtered required check never reports and -# blocks branch protection forever" +# The deletion was gated on a query, recorded in ci.yml beside the removal: +# nothing `needs:` the job, it uploaded no artifact so no consumer could read +# one, and it is not a required context. `gate` had already stopped reading it +# (PP-066 C0-5, PRQ-013, #2982). It cost 150 minutes of two mutation sweeps on a +# clean-room runner on every push to every open PR. # -# Both halves are scars, not style. A workflow-level `paths:` filter means the -# workflow does not run at all on a PR that misses the filter, so a check -# GitHub is told to require never produces a check run: the PR sits PENDING -# forever and nothing can merge — the phantom-required-check deadlock this -# repository has already hit once. A job-level `if:` is the opposite shape: the -# workflow runs, the job reports `skipped`, and `gate` can read that result and -# decide. `mutants` (ci.yml) is the in-repo precedent. +# So the rules invert, and the FALSIFIER SURVIVES THE DELETION — that is the +# whole point of keeping this file rather than deleting it too: # -# Until now that rule lived in a COMMENT, and a comment is not a trigger. The -# same file already carries the receipt of what that costs: check_hardcoded_paths.sh -# was left unwired behind a comment saying to promote it "once the fleet carries -# pmat >= 3.32.0", nothing re-evaluated the condition, and 20 machine-specific -# paths landed through the gap. So the rule is checked here rather than written -# down here. +# R1 ci.yml declares NO job that invokes check_pr_review_receipt.sh. +# This is the standing falsifier. Putting the job back — on +# `pull_request` as it used to be, or on any other event — turns this +# guard RED rather than passing quietly. The case table carries the old +# wiring VERBATIM, asserted FAIL, so the row is about the thing that +# actually happened and not about a shape someone invented. # -# THE FOUR RULES -# -# R1 ci.yml INVOKES scripts/check_pr_review_receipt.sh on a non-comment line. -# check_guards_are_wired.sh already asserts this generically, but through a -# shrink-only BASELINE — and a baseline entry is exactly what someone -# removing this wiring would reach for. Naming the file here means the -# wiring cannot be traded away for a line in a text file. +# Restoring the job deliberately is therefore a code change here plus a +# new case-table row, which is the friction this repository asks for. It +# is not a rule that can be traded away for a line in a baseline file. # # R2 ci.yml declares NO workflow-level `paths:` / `paths-ignore:` filter. -# ci.yml is where BOTH required checks live (`ci / gate` and -# `workspace-test`), so a path filter here is the deadlock, not a -# hypothetical. Scoped to ci.yml and named: check_workflow_path_filters.sh -# governs the *other* workflows, where a filter is legal and the risk is -# going dark instead of deadlocking. -# -# R3 the job that invokes the receipt guard carries a JOB-level `if:`. -# Job-level and step-level are distinguished by indentation, which is what -# makes "job-level" mechanical: a job key sits at 2 spaces, its `if:` at 4, -# and a step's `if:` at 8 or more. A step-level `if:` would leave the JOB -# reporting success on an event where it checked nothing. +# UNCHANGED, and unrelated to the job: ci.yml is where BOTH required +# checks live (`ci / gate`, `workspace-test`), so a path filter here is a +# deadlock — every PR that misses the filter sits PENDING forever. +# check_workflow_path_filters.sh governs the other workflows, where a +# filter is legal and the risk is going dark instead of deadlocking. # -# R4 that `if:` evaluates TRUE on `pull_request` and FALSE on `push`, -# `merge_group` and `workflow_dispatch` — both polarities, because -# "it has an `if:`" is satisfied by `if: false`, which is a gate that -# never runs, and by `if: always()`, which is no gate at all. +# R3 the receipt guard family is still REACHABLE. `check_pr_review_receipt.sh` +# is a tracked `scripts/check_*.sh`, which is exactly guard_tree.sh's +# derived universe (`git ls-files 'scripts/check_*.sh'`, BSE-001 PR-A), so +# it runs in `guard-tree`, a job `gate` needs. Without this rule the +# deletion could be followed by renaming or untracking the guard and +# nothing would notice: R1 would still hold, vacuously, over a family that +# no longer runs anywhere. R1 and R3 are the two halves of one claim — +# the job is gone AND the guards it used to carry still run. # -# THE EVALUATOR IS DELIBERATELY NARROW, AND REFUSES RATHER THAN GUESSES. -# It understands exactly one expression shape — a disjunction of -# `github.event_name == ''` — and any other `if:` is a hard FAILURE -# saying so. That is the correct direction: a guard that silently widens its -# pattern to cover a form it cannot reason about is the defect this repository -# has shipped six times. Extending the evaluator is a code change with a new -# case-table row, not an accident. +# WHAT THIS FILE DOES NOT CLAIM. The two mutation sweeps the job carried +# (`scripts/mutate-guard.sh`, `scripts/mutate_quorum_arm.sh`) and its 43-row bats +# fixture table are not `check_*.sh` and no workflow invokes them, so they run +# nowhere in CI today. That is stated in ci.yml at the deletion site and in the +# PR that removed them; it is deliberately NOT asserted here, because a rule +# nobody can satisfy is the mirror of one that cannot fail. # # bash scripts/check_pr_review_wiring.sh # check # bash scripts/check_pr_review_wiring.sh --self-test # case table, both polarities @@ -67,8 +61,7 @@ # Used by --self-test to drive fixtures; there is no value # of it that turns a check off. # -# EXIT: 0 all four rules hold; 1 anything else, including an `if:` this guard -# cannot evaluate. +# EXIT: 0 all three rules hold; 1 anything else. set -uo pipefail @@ -77,23 +70,13 @@ REPO_ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)" CI_YML="${PR_REVIEW_CI_YML:-$REPO_ROOT/.github/workflows/ci.yml}" GUARD_BASENAME='check_pr_review_receipt.sh' -# Same invocation shape check_guards_are_wired.sh uses: a MENTION is not an -# invocation, and a trailing `#` comment defeated the first version of that -# pattern, so comments are stripped from the first `#` before matching. -# `[.]` and not `\.`: awk warns "escape sequence treated as plain ." on a -# dynamic regex, and a warning on stderr from a guard is how a real diagnostic -# gets scrolled past. GUARD_RE="(^|[[:space:];&|(])((ba)?sh[[:space:]]+|[.]/)?[^[:space:]]*check_pr_review_receipt[.]sh([[:space:]]|$|['\"])" -# Events the workflow can be triggered by, and whether the receipt job must run. -# Driven as a table rather than asserted once: the FALSE rows are what stop -# `if: always()` and a step-level `if:` from reading as compliance. -EVENTS_TRUE='pull_request' -EVENTS_FALSE='push merge_group workflow_dispatch' - # --------------------------------------------------------------------------- -# invoking_job — name of the job whose steps invoke the receipt guard. -# Prints nothing when no job does. +# invoking_job — name of each job whose steps invoke the receipt guard. +# Prints nothing when no job does. A MENTION is not an invocation: comments are +# stripped from the first `#` before matching, because a trailing `#` comment +# defeated the first version of this pattern. # --------------------------------------------------------------------------- invoking_job() { awk -v re="$GUARD_RE" ' @@ -108,26 +91,6 @@ invoking_job() { ' "$1" | LC_ALL=C sort -u } -# --------------------------------------------------------------------------- -# job_level_if — the job-level `if:` expression, verbatim. -# Indentation IS the definition: 4 spaces is the job's own key, 6+ belongs to a -# step. Prints nothing when the job has no job-level `if:`. -# --------------------------------------------------------------------------- -job_level_if() { - awk -v want="$2" ' - /^jobs:[[:space:]]*$/ { injobs = 1; next } - !injobs { next } - /^[^[:space:]#]/ { injobs = 0; next } - /^ [A-Za-z0-9_-]+:/ { cur = $0; sub(/^ /, "", cur); sub(/:.*$/, "", cur); inj = (cur == want); next } - inj && /^ if:[[:space:]]*[^[:space:]]/ { - line = $0 - sub(/^ if:[[:space:]]*/, "", line) - sub(/[[:space:]]+$/, "", line) - print line - } - ' "$1" -} - # --------------------------------------------------------------------------- # workflow_path_filters — every `paths:`/`paths-ignore:` key inside the # top-level `on:` block, as "line: text". Prints nothing when there are none. @@ -144,55 +107,45 @@ workflow_path_filters() { } # --------------------------------------------------------------------------- -# eval_if -# 0 -> the expression is TRUE for that event -# 1 -> the expression is FALSE for that event -# 2 -> this guard cannot evaluate the expression (a FAILURE, never a pass) +# guard_is_tracked — R3. The receipt guard is a tracked scripts/check_*.sh, so +# guard_tree.sh's derived universe runs it. `git ls-files` and not a filesystem +# probe: guard_tree derives its universe the same way, and an untracked file is +# invisible to it however present it looks on disk. # --------------------------------------------------------------------------- -eval_if() { - local expr=$1 ev=$2 norm lits - norm=$(printf '%s' "$expr" | sed 's/[[:space:]][[:space:]]*/ /g; s/^ //; s/ $//') - # A herestring, never `printf ... | grep -q`: on a pipe grep can exit 141 on - # SIGPIPE despite having MATCHED, and this repository has shipped that four - # times in one day. - if ! grep -Eq -- "^github\.event_name == '[a-z_]+'( \|\| github\.event_name == '[a-z_]+')*\$" <<<"$norm"; then - return 2 - fi - lits=$(grep -oE "'[a-z_]+'" <<<"$norm" | tr -d "'" | tr '\n' ' ') - case " $lits " in - *" $ev "*) return 0 ;; - *) return 1 ;; - esac +guard_is_tracked() { + git -C "$REPO_ROOT" ls-files --error-unmatch "scripts/$GUARD_BASENAME" >/dev/null 2>&1 } # --------------------------------------------------------------------------- -# check_file — R1..R4. 0 = all hold. Diagnostics on stdout. +# check_file — R1..R3. 0 = all hold. Diagnostics on stdout. # --------------------------------------------------------------------------- check_file() { - local f=$1 job ifexpr ev filters rc + local f=$1 job filters n if [ ! -f "$f" ]; then printf 'FAIL R0: no workflow at %s\n' "$f" return 1 fi - # R1 + # R1 — the standing falsifier: the receipt job must not be back. job=$(invoking_job "$f") - if [ -z "$job" ]; then - printf 'FAIL R1: no job in %s invokes %s.\n' "$f" "$GUARD_BASENAME" - printf ' A guard named only in a comment is not wired (PRREV-006, spec 9.6).\n' - return 1 - fi - if [ "$(printf '%s\n' "$job" | grep -c .)" -ne 1 ]; then - printf 'FAIL R1: %s is invoked by more than one job:\n' "$GUARD_BASENAME" + if [ -n "$job" ]; then + n=$(printf '%s\n' "$job" | grep -c .) + printf 'FAIL R1: %s job(s) in %s invoke %s:\n' "$n" "$f" "$GUARD_BASENAME" printf '%s\n' "$job" | sed 's|^| |' - printf ' Two jobs means two `if:` conditions to keep in step; R3/R4 would\n' - printf ' then hold for one of them while the other went dark.\n' + printf ' That job was DELETED on 2026-09-08 (BSE-15): 150 minutes of two\n' + printf ' mutation sweeps on a clean-room runner, on every push to every open\n' + printf ' PR, gating nothing — no `needs:`, no artifact, not a required\n' + printf ' context, and `gate` had already stopped reading it (C0-5, #2982).\n' + printf ' The guard itself still runs: it is a scripts/check_*.sh and so is in\n' + printf ' guard_tree.sh'"'"'s derived universe (R3 below).\n' + printf ' Bringing the job back is a deliberate change to this rule plus a new\n' + printf ' case-table row — not an edit to ci.yml alone.\n' return 1 fi - printf 'ok R1 %s is invoked by job `%s`\n' "$GUARD_BASENAME" "$job" + printf 'ok R1 no job in %s invokes %s (the receipt job stays deleted)\n' "$(basename "$f")" "$GUARD_BASENAME" - # R2 + # R2 — unchanged. filters=$(workflow_path_filters "$f") if [ -n "$filters" ]; then printf 'FAIL R2: %s declares a workflow-level path filter:\n' "$f" @@ -204,53 +157,25 @@ check_file() { fi printf 'ok R2 no workflow-level paths:/paths-ignore: filter in %s\n' "$(basename "$f")" - # R3 - ifexpr=$(job_level_if "$f" "$job") - if [ -z "$ifexpr" ]; then - printf 'FAIL R3: job `%s` has no JOB-level `if:` (4-space indent under the job key).\n' "$job" - printf ' A step-level `if:` leaves the JOB reporting success on an event\n' - printf ' where it checked nothing.\n' + # R3 — the other half of R1: deleted job, guards still reachable. + if ! guard_is_tracked; then + printf 'FAIL R3: scripts/%s is not tracked by git.\n' "$GUARD_BASENAME" + printf ' guard_tree.sh derives its universe from\n' + printf " \`git ls-files 'scripts/check_*.sh'\`, so an untracked or renamed\n" + printf ' guard runs NOWHERE — and R1 would keep holding over a family that\n' + printf ' no longer exists. Track it under that name, or move its rules to a\n' + printf ' guard that is tracked and say so here.\n' return 1 fi - if [ "$(printf '%s\n' "$ifexpr" | grep -c .)" -ne 1 ]; then - printf 'FAIL R3: job `%s` has %s job-level `if:` keys; YAML keeps the last and\n' \ - "$job" "$(printf '%s\n' "$ifexpr" | grep -c .)" - printf ' the earlier ones read as enforcement that is not there.\n' - return 1 - fi - printf 'ok R3 job `%s` carries a job-level if: %s\n' "$job" "$ifexpr" + printf 'ok R3 scripts/%s is tracked, so guard_tree.sh runs it\n' "$GUARD_BASENAME" - # R4 — both polarities. - rc=0 - for ev in $EVENTS_TRUE; do - eval_if "$ifexpr" "$ev" - case $? in - 0) printf 'ok R4 %-18s -> runs\n' "$ev" ;; - 1) printf 'FAIL R4: %s -> SKIPPED, but the receipt is addressed to a PR.\n' "$ev"; rc=1 ;; - *) printf 'FAIL R4: this guard cannot evaluate `%s`.\n' "$ifexpr" - printf ' It understands only a disjunction of\n' - printf " github.event_name == ''. Extend the evaluator and add a\n" - printf ' case-table row; do not widen the pattern to make this pass.\n' - return 1 ;; - esac - done - for ev in $EVENTS_FALSE; do - eval_if "$ifexpr" "$ev" - case $? in - 1) printf 'ok R4 %-18s -> skipped\n' "$ev" ;; - 0) printf 'FAIL R4: %s -> runs. There is no PR number on this event, so the\n' "$ev" - printf ' receipt path evidence/pr-review/// has no subject.\n'; rc=1 ;; - *) printf 'FAIL R4: this guard cannot evaluate `%s`.\n' "$ifexpr"; return 1 ;; - esac - done - return "$rc" + return 0 } # --------------------------------------------------------------------------- -# --self-test — must-hold / must-fail rows over synthesized workflows, plus the -# evaluator's own truth table. Every rule gets a mutation that turns it RED and -# a control that must stay GREEN, because "refuse everything" reads green -# otherwise. +# --self-test — must-hold / must-fail rows over synthesized workflows. Every +# rule gets a mutation that turns it RED and a control that must stay GREEN, +# because "refuse everything" reads green otherwise. # --------------------------------------------------------------------------- if [ "${1:-}" = "--self-test" ]; then TD=$(mktemp -d) || exit 1 @@ -270,18 +195,13 @@ if [ "${1:-}" = "--self-test" ]; then } INVOKE=' - run: bash scripts/check_pr_review_receipt.sh tests/fixtures/pr-review/row-14-complete-gpu-review' - JOBIF=" if: github.event_name == 'pull_request'" # assert_file