test(manifest): cover the installer's integrity surface (backend#1729 sweep 6) - #709
Open
LukasWodka wants to merge 2 commits into
Open
test(manifest): cover the installer's integrity surface (backend#1729 sweep 6)#709LukasWodka wants to merge 2 commits into
LukasWodka wants to merge 2 commits into
Conversation
… sweep 6)
Sweep 6 is "mutation-check the existing suites". Measuring first: of the 24
shell scripts under scripts/ (excluding tests), exactly TWO are referenced
by no bats suite -- check-style.sh and gen-manifest.sh. Client is in good
shape; this closes the one that matters most.
gen-manifest.sh produces scripts/manifest.sha256, which install.sh
verifies every fetched sub-script against BEFORE running privileged steps.
It is the installer's integrity surface, and nothing proved its guards
fire.
It carries three claims in its own comments. All three are TRUE -- worth
stating, since this epic is largely a record of such claims being false:
1. gen-manifest's FILES must match install.sh's FILES, "or vice versa"
2. the same for install.ps1's $Files
3. --check is non-zero on drift
What was missing is any test that would notice them ceasing to be true.
Both cross-checks are awk extractions of another file's array literal --
precisely the parser that goes quietly stale when the parsed file is
reformatted. 12 cases now pin them, including both stale-parser cases:
rename install.sh's array and the guard must REFUSE, not compare nothing
to nothing and agree.
ONE REAL GAP FOUND AND FIXED. Emptying FILES makes the manifest cover
nothing, and --check then compared an empty manifest to an empty
regeneration. It did fail -- on `set -u`'s "FILES[@]: unbound variable".
A failure by accident, whose message says nothing about the integrity
surface just lost. Now refused explicitly, naming the consequence: "an
empty manifest verifies nothing".
The first version of that guard used ${#FILES[@]}, which is ITSELF an
unbound-variable error on an empty array under bash 3.2 -- what macOS
ships and what this repo deliberately targets. So the guard reproduced,
by a different route, the exact unintelligible failure it was written to
replace. It now uses ${arr[*]-}, which expands safely whether the array
is unset, empty or populated. Caught by running it on bash 3.2, not by
reading it.
Also in the suite: a case asserting the COMMITTED manifest lists every
file install.sh fetches. That guards the premise rather than the
mechanism -- if the manifest were current AND short, every other test
here could pass while a fetched script had no digest at all.
12/12 pass; bats-hygiene passes, so every assertion is load-bearing
rather than advisory.
Remaining from the measurement: check-style.sh has no suite. Filed rather
than bundled -- one self-contained change per PR.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ndows The "committed manifest lists every file both bootstraps fetch" case could pass vacuously: missing starts at 0 and an empty awk/grep extract never entered the loop, so the assertion held having verified nothing. It also only scraped install.sh's FILES array despite claiming both bootstraps, so a Windows-only fetch absent from manifest.sha256 went unnoticed. Now scrape BOTH bootstraps -- install.sh's FILES and install.ps1's $Files, using the same awk/sed extractions gen-manifest.sh itself uses -- and fail loudly when either extract is empty (a broken scrape means the parser went stale, not that coverage is complete), before the missing-count loop runs. Bugbot, client#709. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Author
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit b9176c1. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sweep 6 is "mutation-check the existing suites". Measuring first: of the 24
shell scripts under scripts/ (excluding tests), exactly TWO are referenced
by no bats suite -- check-style.sh and gen-manifest.sh. Client is in good
shape; this closes the one that matters most.
gen-manifest.sh produces scripts/manifest.sha256, which install.sh
verifies every fetched sub-script against BEFORE running privileged steps.
It is the installer's integrity surface, and nothing proved its guards
fire.
It carries three claims in its own comments. All three are TRUE -- worth
stating, since this epic is largely a record of such claims being false:
What was missing is any test that would notice them ceasing to be true.
Both cross-checks are awk extractions of another file's array literal --
precisely the parser that goes quietly stale when the parsed file is
reformatted. 12 cases now pin them, including both stale-parser cases:
rename install.sh's array and the guard must REFUSE, not compare nothing
to nothing and agree.
ONE REAL GAP FOUND AND FIXED. Emptying FILES makes the manifest cover
nothing, and --check then compared an empty manifest to an empty
regeneration. It did fail -- on
set -u's "FILES[@]: unbound variable".A failure by accident, whose message says nothing about the integrity
surface just lost. Now refused explicitly, naming the consequence: "an
empty manifest verifies nothing".
The first version of that guard used ${#FILES[@]}, which is ITSELF an
unbound-variable error on an empty array under bash 3.2 -- what macOS
ships and what this repo deliberately targets. So the guard reproduced,
by a different route, the exact unintelligible failure it was written to
replace. It now uses ${arr[*]-}, which expands safely whether the array
is unset, empty or populated. Caught by running it on bash 3.2, not by
reading it.
Also in the suite: a case asserting the COMMITTED manifest lists every
file install.sh fetches. That guards the premise rather than the
mechanism -- if the manifest were current AND short, every other test
here could pass while a fetched script had no digest at all.
12/12 pass; bats-hygiene passes, so every assertion is load-bearing
rather than advisory.
Remaining from the measurement: check-style.sh has no suite. Filed rather
than bundled -- one self-contained change per PR.
Co-Authored-By: Claude Opus 5 noreply@anthropic.com
Note
Low Risk
Changes strengthen supply-chain CI checks and installer manifest generation; no runtime auth or data-path behavior changes beyond clearer failure on misconfiguration.
Overview
Adds
scripts/tests/gen-manifest.batsto exercisegen-manifest.sh --checkagainst copiedscripts/trees: lockstep withinstall.shFILESandinstall.ps1$Files, non-zero drift when hashes or files are wrong, and fail-closed behavior when those arrays can’t be parsed.gen-manifest.shnow refuses an emptyFILESorWINDOWS_FILESsurface with a clear integrity message instead of failing indirectly underset -u(or agreeing on an empty manifest during--check). A final test asserts the committedmanifest.sha256lists every path both bootstraps fetch.Reviewed by Cursor Bugbot for commit b9176c1. Bugbot is set up for automated code reviews on this repo. Configure here.