Skip to content

ci(2183): the R8 manifest check lived in a job nobody has to wait for - #755

Merged
LukasWodka merged 3 commits into
developfrom
fix/2183-r8-gate-required
Aug 19, 2026
Merged

ci(2183): the R8 manifest check lived in a job nobody has to wait for#755
LukasWodka merged 3 commits into
developfrom
fix/2183-r8-gate-required

Conversation

@LukasWodka

@LukasWodka LukasWodka commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Summary

The R8 installer-manifest supply-chain check could not block a merge, and could be silenced by an unrelated apt-get.

Static analysis (installer-tests.yaml's static job) is a required status check on no branch — verified against develop, staging, main and both rulesets. Four guards lived there: R8 manifest, cross-OS facts single-sourcing (#435/#616), style/terminology, PSScriptAnalyzer. So a PR could change an installer sub-script, leave scripts/manifest.sha256 stale, and merge on its required checks — publishing a manifest that rejects the very scripts it ships.

They could also be silenced outright. On five consecutive runs of #752 the job's ShellCheck step consumed the whole timeout-minutes: 10 on apt-get update && apt-get install shellcheck:

4. ShellCheck (libs + entrypoints)  cancelled  08:27:28 .. 08:37:38
5. Installer manifest (R8)          skipped
6. Installer facts single-sourced   skipped
7. Style + terminology guard        skipped
8. PSScriptAnalyzer                skipped

Every required check on that sha was green. And the apt-get was never needed: shellcheck ships on ubuntu-latest — the org's quality / shellcheck job installs nothing and prints its version 3s after checkout.

Approach

No branch-protection change, no new required context. Source-of-truth drift is already required on develop and main, and its own header already declares itself "the home for every duplicated-declaration guard in this repo, because it is the one that GATES".

The two lists had already diverged in the way that hides: make drift ran R8 + check-facts + check-style, the required job ran check-drift + env-vocabulary — each side gated exactly what the other did not. A stale manifest was caught by the pre-push hook and not at the merge gate. So there is now one list, in the Makefile, and the required job runs it.

Guard Before After
gen-manifest.sh --check (R8) Static analysis — required nowhere Source-of-truth drift — required on develop + main
check-facts.sh --check Static analysis Source-of-truth drift
check-style.sh Static analysis Source-of-truth drift
bash -n + shellcheck Static analysis, restating Lint's own 19-file sweep Lint via make lint (already there)
PSScriptAnalyzer Static analysis Lint
shellcheck --severity=warning Static analysis, advisory make lint-warnings, still advisory

The static job is deleted — every part of it either moved to a required context or was already duplicated by one.

Guards no longer share a fate. make drift runs all five even when one fails, then exits non-zero. One red guard no longer hides four others, and it still blocks — the guards are independent in their reporting, not in whether they gate. No timeout-minutes on the drift job, deliberately: a required job that hangs blocks the merge, whereas one killed mid-way reports guards as skipped. Prefer the slow red.

Mutation proof

Changed scripts/lib/cluster.sh without regenerating the manifest, asserting the anchor actually applied:

ANCHOR APPLIED: cluster.sh blob 87080bdb -> 0b74919c
drift RED (exit 2) — [ERROR] scripts/manifest.sha256 is out of date.
restored: drift GREEN

Then a stale manifest and a style violation together: all five guards ran, both failures reported (!! FAILED ×2), exit non-zero.

Docs that asserted the gate existed

docs/SUPPLY_CHAIN.md claimed "A CI gate in installer-tests.yaml … fails any PR that changes a sub-script without regenerating the committed manifest". That was false for as long as it was written. .cursor/BUGBOT.md said the same. Both now name the required job, and SUPPLY_CHAIN.md carries a note telling the next person to move the check to a job whose context is in branches/{develop,main}/protection.

Test plan

  • make drift — 5/5 guards green; mutation-proved red twice (above)
  • make lint, make lint-warnings, make check — green
  • bats scripts/tests/*.bats1125/1125 pass
  • All three workflows parse; job/step lists verified via yaml.safe_load
  • Guard scripts confirmed 100755 in git, so the deleted chmod +x steps were redundant
  • No client/** change, so no Chart.yaml bump needed

Checklist

  • Targets develop
  • No secrets, no new permissions, no untrusted interpolation added to any workflow
  • Arms while green — every check green before the guards moved
  • Mutation-proved, with the anchor asserted

Deliberately not fixed

Pester does not gate. Pester (windows-latest) / Pester (ubuntu-latest) are required on no branch, so Unit tests gates bats only and the PowerShell installer has no gating unit suite. Making it required means a required check on a windows-latest runner — a cost/flake decision, not a cleanup. Flagged in standard-checks.yml and in tracebloc/backend#2183.

staging requires neither Lint, Unit tests nor Source-of-truth drift — only gate / gate + the four quality / *. Promotions reach staging via the release train off develop where all three are required, so it is not a hole today, but it is not what the other branches look like.

Refs tracebloc/backend#2183

🤖 Generated with Claude Code


Note

Medium Risk
Changes which CI contexts block merges and centralizes guard logic in the Makefile; mistakes could false-block PRs or briefly miss a guard, but behavior is documented and fail-closed by design.

Overview
Moves R8 manifest, cross-OS facts, style, and related drift guards from installer-tests’ non-required Static analysis job into the already-required Source-of-truth drift job, which now runs a single make drift step instead of enumerating guards in YAML.

Deletes the entire static job from installer-tests.yaml (bash -n, shellcheck, gen-manifest, check-facts, check-style, PSScriptAnalyzer). PSScriptAnalyzer and advisory shellcheck warnings land on required Lint (standard-checks.yml); PSScriptAnalyzer reuses the image’s module when present.

Makefile becomes the sole list of drift guards (DRIFT_GUARDS, including check-drift.sh), with fail-closed execution (empty list, quote-collapse, run-all-then-fail). scripts/sh-files.sh unifies shell file discovery for shellcheck and lint-warnings. drift-checks adds a 20-minute timeout and push paths for k3s-cuda/Makefile; docs/BUGBOT note the real gating job.

Reviewed by Cursor Bugbot for commit 75b042f. Bugbot is set up for automated code reviews on this repo. Configure here.

`Static analysis` (installer-tests.yaml's `static` job) is a required status
check on no branch — measured against develop, staging, main and both rulesets.
Four guards lived there: the R8 installer-manifest check, cross-OS facts
single-sourcing, style/terminology, and PSScriptAnalyzer. A PR could change an
installer sub-script, leave scripts/manifest.sha256 stale, and merge on its
required checks — shipping a manifest that rejects the scripts it covers.

They could also be silenced. On five runs of #752 the job's ShellCheck step ate
the whole `timeout-minutes: 10` on `apt-get install shellcheck`; all four guards
reported `skipped` while every required check went green. The apt-get was never
needed — shellcheck is on the ubuntu-latest image.

No branch-protection change and no new required context: `Source-of-truth drift`
is already required on develop and main, and its header already declares itself
the home for guards that must gate. The Makefile's `drift` target already ran
R8 + check-facts + check-style while that job ran check-drift + env-vocabulary,
so each side gated what the other did not. One list now, in the Makefile.

- drift-checks.yaml runs `make drift` and holds no guard list of its own
- make drift gains check-drift.sh (five guards), and runs ALL of them before
  failing, so one red guard no longer hides the rest — it still blocks
- Lint (required) absorbs the deleted job's bash -n + shellcheck, which restated
  Lint's own 19-file SHELLCHECK_FILES sweep at the same severity, plus its
  PSScriptAnalyzer; `make lint-warnings` keeps the advisory SC2034 pass
- apt-get in Lint is now conditional on shellcheck being absent
- the k3s-cuda paths move to drift-checks.yaml with the check-facts guard (#616)
- docs/SUPPLY_CHAIN.md and .cursor/BUGBOT.md both asserted the gate existed;
  corrected to name the required job

Mutation-proved: changing cluster.sh without regenerating the manifest turns
`make drift` red (anchor asserted: blob 87080bd -> 0b74919c); a stale manifest
plus a style violation reports both and exits non-zero. bats 1125/1125 green.

Refs tracebloc/backend#2183

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@LukasWodka
LukasWodka requested a review from saadqbal as a code owner August 19, 2026 10:05
@LukasWodka LukasWodka self-assigned this Aug 19, 2026
@LukasWodka

Copy link
Copy Markdown
Contributor Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 0172f7e. Configure here.

@aptracebloc

Copy link
Copy Markdown
Contributor

Findings from a review pass. No verdict attached.

The thesis holds up — I read branch protection independently rather than taking the body on trust, and every claim in it checks out (see the bottom section). The findings are about two things the move carries along with it, plus the collision with #754.

#755 × #754 — the dangerous breakage merges without a conflict marker

Verified by running a real 3-way git merge-file from the merge base (8de5d64) against both heads. Visible conflicts exist in all three shared files — Makefile (a timing comment), standard-checks.yml (the Lint step block), installer-tests.yaml (the header list, and the static job: this PR deletes it, #754 edits it).

The problem is what merges cleanly. #754 deletes SHELLCHECK_FILES; this PR's new Makefile:178@shellcheck --severity=warning --shell=bash $(SHELLCHECK_FILES) || true — then expands to a bare invocation with no file operands, and || true swallows whatever shellcheck says about that. The advisory SC2034 sweep this PR goes to the trouble of preserving becomes a permanent no-op reporting green. Different offsets, so git produces no marker; the only signal is a conflict about a benchmark number.

The other order is worse: #754's diff replaces the whole Lint step region and, as written, would drop the PSScriptAnalyzer step this PR just added — silently un-gating the PowerShell lint of scripts/install.ps1, which is half the point of the move.

Even in the better order (#754 first), this PR's rebase needs Ensure shellcheck dropped and run: make lintmake parse (after #754, make lint pulls in its new shellcheck target, reintroducing the third run #754 exists to remove), and lint-warnings rewritten onto the derived set. No coverage hole from the stale prose, since quality / shellcheck is required on both develop and main.

Findings

1. standard-checks.yml:112-117 — the corrected premise is carried across unfixed

Install-Module PSScriptAnalyzer -Force -SkipPublisherCheck is an unconditional PSGallery network fetch, newly placed inside a required, timeout-less job. This PR's own run proves the install is both unnecessary and slow: the log reads WARNING: The version '1.25.0' of module 'PSScriptAnalyzer' is currently in use — it's already on the runner image — and the step took 38s of the job's 59s.

That's the identical wrong premise this PR corrects two steps above for shellcheck ("shellcheck IS on the ubuntu-latest runner image"), promoted from a bounded advisory job into an unbounded required one. If PSGallery has a bad day, required Lint sits pending toward GitHub's 360-minute default, blocking every merge to develop and main. Same shape as the shellcheck fix: if (-not (Get-Module -ListAvailable PSScriptAnalyzer)) { Install-Module ... }.

2. standard-checks.yml:41-43Lint has no timeout-minutes

Pre-existing — confirmed against ref=develop that it never did, so this PR doesn't introduce it. But adding a 38-second network install to that job makes the missing bound matter in a way it didn't before. All seven surviving jobs in installer-tests.yaml have bounds (10/20/30m); the two required jobs this PR routes everything through have none.

3. drift-checks.yaml:95-97 — the no-timeout reasoning doesn't survive this PR's own change

The comment argues that a job killed mid-way reports guards as skipped, so an unbounded job is preferable. That was true because Static analysis was not a required context — the cancellation was invisible to branch protection. Now that Source-of-truth drift is required, a timeout-kill sets that context to cancelled, which branch protection treats as not-success: the PR goes red and stays blocked. The failure mode being guarded against is already closed by making the context required, so the tradeoff described no longer exists.

A bounded timeout-minutes: 20 would give both properties, against guards that measured 0.9s in CI. Worth noting the job's real hang risk isn't make drift at all — it's azure/setup-helm at :60-63, which downloads a binary before any guard runs. (#754 takes the opposite position and adds bounds to both jobs; the two need settling either way.)

4. docs/SUPPLY_CHAIN.md:109-110 — nit, and it defeats the note's own purpose

The runbook tells the next person to move the check "to a job whose context is in branches/{develop,main}/protection". That endpoint requires admin and returns a bare {"message":"Not Found"} at maintain level — I hit exactly this. The 404 is indistinguishable from "branch is not protected", so a reader following the instruction concludes the opposite of the truth. The endpoint that works for the rest of the team is repos/tracebloc/client/branches/{branch}.protection.required_status_checks.contexts. Worth naming that one, since the point of the note is that it be verifiable.

5. standard-checks.yml:93,113 — nit

if: always() also fires on job cancellation, so a cancelled run still kicks off the PSGallery fetch. if: ${{ !cancelled() }} expresses the stated intent ("a red make lint above still reports the PowerShell verdict") precisely.

6. PR body — nit

"No scripts/ file changed, so no regeneration" is wrong as written — scripts/check-style.sh is under scripts/. The conclusion is right, and I verified it rather than assuming: gen-manifest.sh's FILES (17 entries: install-k8s.sh + lib/*.sh) and WINDOWS_FILES cover only bootstrap-fetched sub-scripts; check-style.sh is in neither, and the committed manifest has 18 lines matching exactly that set. The accurate wording is "no bootstrap-fetched script changed."

Verified and clean

Branch protection, read independently (via repos/tracebloc/client/branches/{b}.protection, since the /protection endpoint 404s at maintain level):

  • develop: Unit tests, Lint, quality / gitleaks, quality / house-rules, quality / shellcheck, quality / action-pins, version-bump-gate / version-check, Source-of-truth drift
  • main: gate / gate, Unit tests, Lint, quality / gitleaks, quality / house-rules, quality / shellcheck, Source-of-truth drift, quality / action-pins
  • staging: gate / gate, quality / gitleaks, quality / house-rules, quality / shellcheck, quality / action-pins

Every claim in the body checks out: Source-of-truth drift and Lint required on develop + main, Static analysis in no list, Pester nowhere, and staging requiring none of the three.

Context names survive the diff, verified empirically rather than by reading YAML — drift: / name: Source-of-truth drift, no matrix, reporting green on this head sha; same for Lint. No orphaned required context.

The paths: trap isn't presentdrift-checks.yaml:47-48's pull_request carries branches: only. The new docker/k3s-cuda/**, build-k3s-cuda.yaml, Makefile entries land on the push: trigger, which is correct. The 20-line header comment documenting that hazard by name is doing its job.

make drift semantics reproduced verbatim in a scratch harness with a failing guard and a missing guard, forced under SHELL=/bin/dash: all five run, both failures print !! FAILED, exits non-zero. IFS='|' splits without whitespace-splitting the multi-word commands; sh -c returns 127 for a missing script, so the fail-closed claim is accurate.

Every deleted static step has a home. The inline shellcheck list extracted from the deleted job is character-for-character the same 19 entries as Makefile:21-40, and the bash -n scope is identical on both sides, so make lint loses nothing. The two chmod +x steps really were redundant — 100755 confirmed on all five guard scripts via the git tree API. No dangling needs: static.

— drafted with Claude Code

@LukasWodka
LukasWodka requested review from aptracebloc and removed request for saadqbal August 19, 2026 11:47
develop (client#747) made telemetry-vocabulary-agreement.sh a REQUIRED guard
(backend#1907) via individual steps + the enumerated lists this PR collapses.
Folded it into DRIFT_GUARDS so `make drift` (the required Source-of-truth
check) still runs it; verified `make drift` green including that guard.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
LukasWodka added a commit that referenced this pull request Aug 19, 2026
…review)

Both findings from Arturo's review. Built on top of b4ab27b (a concurrent
session's develop merge) rather than a second resolution of my own -- see the
note at the end, which is the more useful half of this message.

FIX 1 -- `make parse` failed open, and #753 had just promoted it to be the
entire content of the required `Lint` check. `find scripts ... | xargs -0 -n1
bash -n` loses find's status across the pipe (recipes run under dash, no
pipefail) and xargs with no input runs nothing and exits 0, so in a tree with no
scripts/ dir it printed

    find: scripts: No such file or directory
    all shell scripts parse

and exited 0, under both shells. It now materialises the list, refuses a zero
count, and prints the derived count. Proven four ways: missing dir -> non-zero,
zero .sh files -> non-zero, real parse error -> non-zero, clean tree -> "all 39
shell scripts parse". The first attempt at that third anchor was INERT --
`if [ -z "$x" ; then` is valid bash, the missing bracket being a runtime not a
parse error -- so it was redone with an unterminated `if`, after checking that
bash -n rejects it standalone.

FIX 2 -- the superset claim was true set-wise and false scope-wise. The deleted
jobs swept their 34 files on every run; `quality / shellcheck` reads only the PR
diff, its caller passes all-files: false and declares no schedule. So #753 as
written left NO CI job doing a whole-tree error-severity sweep -- demonstrable
on #754 itself, where that job is green in 8s reading "Shell files to check: 0".
`Lint` therefore runs `make lint` (parse + derived whole-tree shellcheck) rather
than `make parse`. Costs no install, shellcheck being preinstalled is the whole
finding, and it restores `make lint` == CI `Lint` parity (backend#1850). The two
survivors are complementary: whole-tree here, diff-scoped in the org job,
neither installing anything, neither holding a file list.

Verified green on the 44-file derived set under the RUNNER's shellcheck 0.9.0
via koalaman/shellcheck:v0.9.0, not merely a local 0.11.0, before arming it on a
required check.

WHY THIS SITS ON b4ab27b. I had resolved the same develop conflict locally and
resolved the Makefile with `git checkout --ours`, which takes the whole file and
so silently DROPPED develop's non-conflicting addition of
`scripts/tests/telemetry-vocabulary-agreement.sh` to the `drift` target. Nothing
would have failed: the guard would simply have stopped running. b4ab27b kept it,
so that resolution is the base and mine was discarded. This is the same shape as
the #755 hazard flagged on the PR -- the damage from a merge is in what
disappears without a marker, not in what conflicts -- and it caught me one hour
after I wrote that warning.

The conflict itself is the thesis: to add one script, #747 had to hand-edit the
enumerated list in three places. The derivation picked it up with no edit, 42 ->
44 on its own.

Refs #753
@LukasWodka

Copy link
Copy Markdown
Contributor Author

Sequencing note from #754's review (Arturo's finding, independently reproduced). This PR and #754 break each other in a way that merges without a conflict marker.

Real 3-way git merge-file from the merge base 8de5d64 against both heads produces exactly one Makefile conflict, and it is the check timing comment (~5 s vs ~10 s). What merges cleanly is the problem:

Different offsets, so git merges them silently. I resolved only the visible seconds comment and ran the result:

$ make lint-warnings
No files specified.
Usage: shellcheck [OPTIONS...] FILES...
$ echo $?
0

shellcheck gets zero file operands and || true swallows the usage error, so the advisory SC2034 sweep this PR goes to the trouble of preserving becomes a permanent no-op that reports green — with nothing but a comment about a benchmark number to warn anyone. That is the same "classifying zero files must FAIL, not report green" mode #754's new shellcheck target is written to prevent.

Reverse order is worse: #754's diff replaces the whole Lint step region, and with static already deleted here, a resolution taking #754's side removes the PSScriptAnalyzer step — un-gating the PowerShell lint of scripts/install.ps1, the R8 Windows trust root.

Suggested order: #754 first, then rebase this one with

  • lint-warnings rewritten onto the derived file set (no SHELLCHECK_FILES),
  • make lint kept as-is — since 80671e8, Standard checks / Lint runs make lint, so that target is CI's definition now, not a local convenience,
  • the conditional Ensure shellcheck install dropped: shellcheck is preinstalled on ubuntu-latest (0.9.0, measured in run 32232682350), which is the whole basis of ci: one shellcheck gate, and it is the one that does not apt-install (#753) #754.

Either way the second PR needs a real rebase and re-review. Please don't resolve this through GitHub's conflict editor — the breakage is outside the markers, so a clean-looking resolution is exactly what produces the silent no-op.

I have not touched this branch. Flagging only.

For the record, this bit me too within the hour: resolving #754's own conflict with develop I used git checkout --ours on the Makefile and silently dropped develop's non-conflicting addition of scripts/tests/telemetry-vocabulary-agreement.sh to the drift target. Nothing failed — the guard would just have stopped running. Same class, same day.

One scoping correction: this PR is not an alternative remedy to backend#2182 / #2183. Those are about making the R8 manifest step a gate, which #754 deliberately leaves alone. The only overlap is where shellcheck lives and whether its file list is enumerated.

@saadqbal saadqbal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The gate is real now — Source-of-truth drift and Lint keep their exact names, both run on every PR with no paths: filter, nothing carries continue-on-error, and no needs: pointed at the job you delete. R8 ends up behind a context that actually blocks. Good change.

One thing I'd fix first: make drift can print drift: all guards green and exit 0 having run zero guards. Inline.

Also note the head moved to 74eeb05 (develop merged in) and brought a sixth guard, telemetry-vocabulary-agreement.sh. Neither prose copy of the guard list followed it — which is the exact divergence this PR exists to stop, so worth catching on the way past.

Not inline because the lines aren't in the diff: Makefile:19 and Makefile:151 still describe SHELLCHECK_FILES and lint in terms of the static job you delete here.

On whether the check still detects the right failures: gen-manifest.sh --check diffs a full regeneration against the committed manifest, so a changed digest and a manifest that has gained or lost a line both fail, and it cross-checks its FILES/WINDOWS_FILES against both bootstraps' arrays with a bats test asserting the manifest lists every fetched file. What nothing checks is a new scripts/lib/foo.sh that install-k8s.sh sources but that never got added to install.sh's FILES — that file is neither fetched nor covered, and every guard stays green. Pre-existing, not yours.

Coordination with #754, which is open on the same three files: it deletes SHELLCHECK_FILES, which your new lint-warnings expands. If it lands first, make lint-warnings becomes a bare shellcheck with no operands behind a || true — permanently green, silently. Whoever rebases second should hand-check that and the Lint step block rather than trust a clean merge.

Ran make drift against this head in a scratch copy: 6/6 green, 2.6 s. Local bats didn't finish inside 10 minutes so I dropped it; CI's bats (bash unit, mocked) is green.

Pre-merge, since I can't read branch protection: confirm Static analysis really is in no required-check list before this merges — if it is required anywhere, deleting the job leaves those PRs waiting on a context that will never report.

Skipping what aptracebloc already raised (the PSGallery install, the missing timeout-minutes, if: always() vs !cancelled()).

Comment thread Makefile Outdated
Comment thread .github/workflows/drift-checks.yaml Outdated
Comment thread Makefile Outdated

@aptracebloc aptracebloc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 74eeb059. The branch has only merged develop since my last pass — no findings addressed — and there's a new fail-open in the required check this PR creates. The thesis is still right; the implementation needs another turn.

First, correcting myself from my earlier comment: I described this and #754 as opposite remedies for the same gap. That was wrong. backend#2182/#2183 are about making the R8 step a gate, which #754 explicitly leaves alone. This PR is a different answer on where shellcheck lives, not the opposite of that one.

N3 (blocking) — make drift can report green having run nothing

@saadqbal found this; I reproduced both shapes independently against this head:

$ make drift DRIFT_GUARDS=
drift: all guards green                → exit 0

$ make drift "DRIFT_GUARDS=bash -c 'exit 1'|scripts/gen-manifest.sh --check"
/bin/sh: 1|scripts/gen-manifest.sh --check: No such file or directory
drift: all guards green                → exit 0

@guards='$(DRIFT_GUARDS)' interpolates inside single quotes, so any future guard containing a 'bash -c '…', python3 -c '…' — terminates the assignment and the list collapses to zero iterations. The newly-required Source-of-truth drift then goes green having run nothing.

The asymmetry is what makes this worth blocking on rather than filing: parse and shellcheck in #754 both carry exactly the zero-count guard drift lacks — same author, same day, same class of defect, opposite outcome. And this PR's whole argument is that a guard which can't fail isn't a gate. Fix: count iterations, compare against the number of |-separated entries, and stop interpolating into a quoted string.

Four findings from my last pass, all untouched

Unconditional Install-Module PSScriptAnalyzer (standard-checks.yml:117) — still unconditional, still inside a required job. The run log shows the module is already on the image (version '1.25.0' … currently in use) and the step took 38s of 59s. Note the pair interaction: #754 adds timeout-minutes: 10 to that exact job, so if #754 lands first this degrades from a 360-minute pending to a bounded 10-minute red. Better, but it's still a network fetch that doesn't need to happen.

Lint has no timeout-minutes — unaddressed here; #754 supplies it. So the pair fixes it, this PR alone doesn't.

drift-checks.yaml:95-97 still reads "No timeout-minutes here, deliberately." The reasoning doesn't survive this PR's own change: guards reported skipped harmlessly only because Static analysis wasn't required. Once Source-of-truth drift is required, a timeout-kill sets it cancelled, which branch protection treats as not-success — red and blocked, which is what you want. timeout-minutes: 20 gives both properties against guards that measure ~1s. The real hang risk isn't make drift anyway; it's azure/setup-helm at :61, which downloads a binary before any guard runs.

docs/SUPPLY_CHAIN.md:106-107 still names branches/{develop,main}/protection. I re-hit the 404 today: that path returns {"message":"Not Found","status":"404"} at non-admin level, indistinguishably from "branch is not protected", while repos/tracebloc/client/branches/develop.protection.required_status_checks.contexts works. A reader following the runbook concludes the opposite of the truth, which defeats the note's own purpose.

N4 (nit) — the count has already drifted

The job comment enumerates five guards and the Makefile says "All five" and "the five drift guards"; DRIFT_GUARDS has been six since the develop merge. Also confirming @saadqbal's sub-point: "all five are pure local file comparisons and all have a --write / regenerate mode" isn't true of check-drift.sh, which the very next sentence admits shells out to helm template. Worth fixing mainly because the job header eight lines above says this job "holds no list of its own" — and this comment is a list of its own that has already drifted.

The $(SHELLCHECK_FILES) no-op is still live

Merged 9d811b39 × 74eeb059, resolved only the one visible Makefile conflict (a timing comment), ran it with an argv-logging stub:

$ make lint-warnings
STUB shellcheck argv: --severity=warning --shell=bash
No files specified.
EXIT=0

SHELLCHECK_FILES has no definition in the merged Makefile — only a mention inside a comment — while lint-warnings still references it. Zero operands, || true swallows the usage error, exit 0. A permanent green no-op, with a benchmark-number conflict as the only warning. make lint on the same tree is healthy (39 parsed, 44 shellcheck operands), so this is specifically the advisory sweep going dark.

Good news on the other half: the PSScriptAnalyzer-drop hazard I flagged earlier is gone. #754 changing run: make parserun: make lint made that step byte-identical on both sides, so the standard-checks.yml conflict shrank to the Install shellcheck region and this PR's advisory-shellcheck and PSScriptAnalyzer steps both merge in cleanly. Verified in the merged file.

Suggested path

Land #754 first — it's green, self-contained, and leaves the R8-gate question open for this PR. Then rebase this onto develop and, while in there:

  • drop Ensure shellcheck (standard-checks.yml:59); preinstalled is #754's whole basis
  • keep run: make lint as-is — now identical on both sides
  • rewrite lint-warnings onto the derived set with the same zero-count guard, or drop || true. Best shape: factor the classifier out of shellcheck: into one place both targets read, so the warning sweep can't silently diverge from the error sweep again
  • fix N3's zero-iteration fail-open and N4's counts
  • settle the drift-checks timeout and the SUPPLY_CHAIN endpoint

Resolve the three conflicts hunk-by-hunk, not whole-file and not in GitHub's conflict editor — both the $(SHELLCHECK_FILES) no-op and #754's new bounded bats step sit outside the markers.

— drafted with Claude Code

LukasWodka added a commit that referenced this pull request Aug 19, 2026
…753) (#754)

shellcheck ran in three places. Two apt-installed it and hung 35+ minutes on
2026-08-19; the third, the org reusable `quality / shellcheck`, passed in ~5 s
because shellcheck is preinstalled on ubuntu-latest (0.9.0). The comment
justifying the install -- "shellcheck itself is not on the runner image" -- was
false.

Measured before deleting: the two enumerated lists were byte-identical to each
other, expanded to 34 files, and were a strict subset of the derived set. The
enumeration had drifted past eight real scripts, and a parse error planted in
index-invariants.sh was invisible to it. Both local copies are gone; the Makefile
now derives the set instead of listing it.

Scope is stated honestly rather than overclaimed: `quality / shellcheck` reads
only the PR diff, so `Standard checks / Lint` runs `make lint` over the whole
tree. Two complementary required gates, neither installing anything, neither
holding a file list. Verified green under the runner's 0.9.0, not just a local
0.11.0.

Also: `make parse` now fails closed (it exited 0 on a missing scripts/ dir while
being the entire content of a required check), both bats installs are bounded and
retried after one stalled on this very PR, every job in standard-checks.yml has a
timeout, and four comments that had gone false were fixed -- file counts removed
from prose entirely, since they rotted twice in one day.

Review by @aptracebloc and @saadqbal found three real defects in this PR, all
fixed here. Follow-ups deliberately left open: backend#2182 / #2183 (make the R8
manifest gate required) and #755's rebase, whose `lint-warnings` will silently
no-op against the deleted SHELLCHECK_FILES if it is merged without one.

Closes #753
…ings

#754 landed on develop, so the sequencing Arturo recommended is satisfied: it
removed shellcheck's third copy and explicitly left the R8-as-a-gate question to
this PR. Three files conflicted; resolved hunk-by-hunk, taking develop's derived
classifier and this PR's deletion of the `static` job.

N3 (blocking, Asad + Arturo) — `make drift` could report green having run zero
guards, in the very check this PR makes required:

  $ make drift DRIFT_GUARDS=                               -> green, exit 0
  $ make drift "DRIFT_GUARDS=bash -c 'exit 1'|scripts/..." -> green, exit 0

`guards='$(DRIFT_GUARDS)'` expanded inside single quotes, so the first guard
containing a `'` terminated the assignment and the list collapsed. Fixed at the
root by exporting the list and reading it from the environment — no quote in a
guard can collapse an env var — plus an empty-list refusal and an iteration
count asserted against the number of declared entries. Both shapes now exit
non-zero; the quoted case runs both guards and reports the failing one. The
comment that claimed "this is NOT the fail-open shape" is now three machine
checks instead of a claim (backend#1729 rule 7).

The `$(SHELLCHECK_FILES)` no-op (Arturo) — #754 deleted that variable while this
PR's new `lint-warnings` still referenced it, so the advisory sweep ran with no
operands and `|| true` swallowed the usage error: permanently green. Confirmed on
the merged tree. Both sweeps now read ONE derivation, scripts/sh-files.sh, which
exits non-zero on a zero-file classification — so a broken derivation is loud in
both instead of silent in either. Mutation-proved via SH_FILES= stub: both
targets go red.

Also from the reviews:
- drift-checks.yaml: deleted the five-guard enumeration. It was already wrong
  (six since telemetry-vocabulary-agreement.sh arrived) eight lines under a
  header claiming the job "holds no list of its own". `make drift` prints the
  live set and count.
- drift-checks.yaml: `timeout-minutes: 20`. The "no timeout, deliberately"
  argument didn't survive this PR — `skipped` guards were harmless only while
  the job was required nowhere; now a kill is `cancelled`, which blocks. The
  hang risk is azure/setup-helm, not the guards.
- standard-checks.yml: dropped `Ensure shellcheck` (preinstalled — #754's basis).
- standard-checks.yml: PSScriptAnalyzer installs only if absent. 1.25.0 ships on
  the image and was already loaded while the step spent 38s of 59s re-fetching
  it inside a required check.
- standard-checks.yml: `if: always()` -> `if: !cancelled()`, so a lost job stops
  spending its budget.
- docs/SUPPLY_CHAIN.md: the runbook said to read `branches/{b}/protection`, which
  is admin-only and 404s indistinguishably from "not protected" — a non-admin
  following it concluded the opposite of the truth. Now names
  `branches/develop` -> `.protection.required_status_checks.contexts`.
- Makefile header: names drift-checks.yaml, and `drift` as the second target
  this file defines rather than copies.

Answering Asad's pre-merge ask: `Static analysis` is in NO required-check list —
develop, staging and main all read, plus both rulesets. Deleting the job leaves
no PR waiting on a context that will never report.

Verified: make check green; shellcheck 45 files (the new script included, and
mutation-proved covered — a planted parse error reddens the gating sweep);
lint-warnings reports 45 and real warnings instead of "No files specified";
drift 6/6 green. bats runs past the local 10-minute budget, as Asad also found;
CI is the authority.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@LukasWodka

Copy link
Copy Markdown
Contributor Author

Both reviews addressed at 75b042f. @saadqbal @aptracebloc — thank you, N3 was real and it was the worst possible defect for this particular PR to ship.

#754 has merged, so the sequencing @aptracebloc recommended is satisfied rather than skipped: develop now carries the derived classifier, and #754's own comment hands the R8-as-a-gate question to this PR. I merged develop and resolved the three conflicts hunk-by-hunk (not whole-file, not in the web editor) — and both of the things you warned sit outside the markers were there.

N3 — fixed at the root, not just detected

The interpolation was the mechanism, so a count alone would have papered over it. The list is now exported and read from the environment, where a ' is just a byte; on top of that an empty list is refused, and the loop asserts its iteration count against the number of declared entries.

$ make drift DRIFT_GUARDS=                                     exit 2
$ make drift "DRIFT_GUARDS=bash -c 'exit 1'|scripts/gen-..."   exit 2  (runs BOTH guards)
$ make drift                                                   exit 0  "all 6 guards green"

The SHELLCHECK_FILES no-op — confirmed and fixed

Reproduced on the merged tree exactly as described: no operands, No files specified., || true, exit 0. Fixed the way you suggested — the classifier is factored into one place both sweeps read, scripts/sh-files.sh, which exits non-zero on a zero-file classification. The advisory half now inherits the gating half's fail-closed property instead of needing its own copy of it.

Mutation-proved with a stub (make <target> SH_FILES=/path/to/stub, overridable only so this path is testable without editing the real classifier):

before after
make shellcheck w/ broken derivation exit 2
make lint-warnings w/ broken derivation exit 0, silent exit 2
make lint-warnings normal No files specified. exit 0 45 file(s) + real findings

The new script is itself classified (45 vs 44) and mutation-proved covered: a planted parse error reddens the gating sweep. It caught a defect in itself on the way in — a comment line beginning with the linter's name is inline-directive syntax, so it was an SC1073 error. Exactly the trap code-quality.yml's own comment warns about.

The other four, all addressed

  • PSGallery install — conditional. 1.25.0 ships on the image and was already loaded while the step spent 38s of 59s re-fetching it inside a required check.
  • Lint timeout — arrives with ci: one shellcheck gate, and it is the one that does not apt-install (#753) #754's merge (timeout-minutes: 10), so the pair does fix it and I am no longer relying on that.
  • drift-checks timeouttimeout-minutes: 20. Your reasoning is the part I had backwards, and it is now written down that way.
  • SUPPLY_CHAIN endpoint — replaced. Verified the asymmetry: branches/develop/protection is admin-only; branches/develop then .protection.required_status_checks.contexts returns the same list at plain-member level. A runbook whose 404 is indistinguishable from "not protected" defeats its own purpose.
  • if: always() to !cancelled() on both steps.

@saadqbal's pre-merge ask — answered with the reads

Static analysis is in no required-check list:

develop: Unit tests, Lint, quality/{gitleaks,house-rules,shellcheck,action-pins},
         version-bump-gate/version-check, Source-of-truth drift
main:    gate/gate, Unit tests, Lint, quality/{gitleaks,house-rules,shellcheck,
         action-pins}, Source-of-truth drift
staging: gate/gate, quality/{gitleaks,house-rules,shellcheck,action-pins}
rulesets: "R8 trust root - protect v* release tags" (tag),
          "promotion-branches-merge-commit-only" (branch) - neither adds a context

So deleting the job leaves no PR waiting on a context that will never report.

Two things I am deliberately not fixing here

@saadqbal's uncovered-lib gap — a new scripts/lib/foo.sh that install-k8s.sh sources but that never reaches install.sh's FILES is neither fetched nor covered, and every guard stays green. You called it pre-existing and I agree; it is a hole in R8's coverage rather than in its gating, so it wants its own ticket and its own mutation test. Filing it separately.

Pester still gates nothingPester (windows-latest) / (ubuntu-latest) are required on no branch, so Unit tests covers bats only. Making it required means a required check on a windows-latest runner: a cost/flake decision, not a cleanup.

bats does not finish inside 10 minutes locally for me either — CI's bats (bash unit, mocked) is the authority.

@LukasWodka

Copy link
Copy Markdown
Contributor Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 75b042f. Configure here.

@LukasWodka
LukasWodka merged commit a7b36ef into develop Aug 19, 2026
46 checks passed
@LukasWodka
LukasWodka deleted the fix/2183-r8-gate-required branch August 19, 2026 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants