Skip to content

test(manifest): cover the installer's integrity surface (backend#1729 sweep 6) - #709

Open
LukasWodka wants to merge 2 commits into
developfrom
ci/1729-sweep6-manifest-suite
Open

test(manifest): cover the installer's integrity surface (backend#1729 sweep 6)#709
LukasWodka wants to merge 2 commits into
developfrom
ci/1729-sweep6-manifest-suite

Conversation

@LukasWodka

@LukasWodka LukasWodka commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Sweep 6 is "mutation-check the existing suites". Measuring first: of the 24
shell scripts under scripts/ (excluding tests), exactly TWO are referenced
by no bats suite -- check-style.sh and gen-manifest.sh. Client is in good
shape; this closes the one that matters most.

gen-manifest.sh produces scripts/manifest.sha256, which install.sh
verifies every fetched sub-script against BEFORE running privileged steps.
It is the installer's integrity surface, and nothing proved its guards
fire.

It carries three claims in its own comments. All three are TRUE -- worth
stating, since this epic is largely a record of such claims being false:

  1. gen-manifest's FILES must match install.sh's FILES, "or vice versa"
  2. the same for install.ps1's $Files
  3. --check is non-zero on drift

What was missing is any test that would notice them ceasing to be true.
Both cross-checks are awk extractions of another file's array literal --
precisely the parser that goes quietly stale when the parsed file is
reformatted. 12 cases now pin them, including both stale-parser cases:
rename install.sh's array and the guard must REFUSE, not compare nothing
to nothing and agree.

ONE REAL GAP FOUND AND FIXED. Emptying FILES makes the manifest cover
nothing, and --check then compared an empty manifest to an empty
regeneration. It did fail -- on set -u's "FILES[@]: unbound variable".
A failure by accident, whose message says nothing about the integrity
surface just lost. Now refused explicitly, naming the consequence: "an
empty manifest verifies nothing".

The first version of that guard used ${#FILES[@]}, which is ITSELF an
unbound-variable error on an empty array under bash 3.2 -- what macOS
ships and what this repo deliberately targets. So the guard reproduced,
by a different route, the exact unintelligible failure it was written to
replace. It now uses ${arr[*]-}, which expands safely whether the array
is unset, empty or populated. Caught by running it on bash 3.2, not by
reading it.

Also in the suite: a case asserting the COMMITTED manifest lists every
file install.sh fetches. That guards the premise rather than the
mechanism -- if the manifest were current AND short, every other test
here could pass while a fetched script had no digest at all.

12/12 pass; bats-hygiene passes, so every assertion is load-bearing
rather than advisory.

Remaining from the measurement: check-style.sh has no suite. Filed rather
than bundled -- one self-contained change per PR.

Co-Authored-By: Claude Opus 5 noreply@anthropic.com


Note

Low Risk
Changes strengthen supply-chain CI checks and installer manifest generation; no runtime auth or data-path behavior changes beyond clearer failure on misconfiguration.

Overview
Adds scripts/tests/gen-manifest.bats to exercise gen-manifest.sh --check against copied scripts/ trees: lockstep with install.sh FILES and install.ps1 $Files, non-zero drift when hashes or files are wrong, and fail-closed behavior when those arrays can’t be parsed.

gen-manifest.sh now refuses an empty FILES or WINDOWS_FILES surface with a clear integrity message instead of failing indirectly under set -u (or agreeing on an empty manifest during --check). A final test asserts the committed manifest.sha256 lists every path both bootstraps fetch.

Reviewed by Cursor Bugbot for commit b9176c1. Bugbot is set up for automated code reviews on this repo. Configure here.

… sweep 6)

Sweep 6 is "mutation-check the existing suites". Measuring first: of the 24
shell scripts under scripts/ (excluding tests), exactly TWO are referenced
by no bats suite -- check-style.sh and gen-manifest.sh. Client is in good
shape; this closes the one that matters most.

gen-manifest.sh produces scripts/manifest.sha256, which install.sh
verifies every fetched sub-script against BEFORE running privileged steps.
It is the installer's integrity surface, and nothing proved its guards
fire.

It carries three claims in its own comments. All three are TRUE -- worth
stating, since this epic is largely a record of such claims being false:

  1. gen-manifest's FILES must match install.sh's FILES, "or vice versa"
  2. the same for install.ps1's $Files
  3. --check is non-zero on drift

What was missing is any test that would notice them ceasing to be true.
Both cross-checks are awk extractions of another file's array literal --
precisely the parser that goes quietly stale when the parsed file is
reformatted. 12 cases now pin them, including both stale-parser cases:
rename install.sh's array and the guard must REFUSE, not compare nothing
to nothing and agree.

ONE REAL GAP FOUND AND FIXED. Emptying FILES makes the manifest cover
nothing, and --check then compared an empty manifest to an empty
regeneration. It did fail -- on `set -u`'s "FILES[@]: unbound variable".
A failure by accident, whose message says nothing about the integrity
surface just lost. Now refused explicitly, naming the consequence: "an
empty manifest verifies nothing".

The first version of that guard used ${#FILES[@]}, which is ITSELF an
unbound-variable error on an empty array under bash 3.2 -- what macOS
ships and what this repo deliberately targets. So the guard reproduced,
by a different route, the exact unintelligible failure it was written to
replace. It now uses ${arr[*]-}, which expands safely whether the array
is unset, empty or populated. Caught by running it on bash 3.2, not by
reading it.

Also in the suite: a case asserting the COMMITTED manifest lists every
file install.sh fetches. That guards the premise rather than the
mechanism -- if the manifest were current AND short, every other test
here could pass while a fetched script had no digest at all.

12/12 pass; bats-hygiene passes, so every assertion is load-bearing
rather than advisory.

Remaining from the measurement: check-style.sh has no suite. Filed rather
than bundled -- one self-contained change per PR.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@LukasWodka
LukasWodka requested a review from saadqbal as a code owner August 13, 2026 12:53
@LukasWodka LukasWodka self-assigned this Aug 13, 2026
Comment thread scripts/tests/gen-manifest.bats
…ndows

The "committed manifest lists every file both bootstraps fetch" case could
pass vacuously: missing starts at 0 and an empty awk/grep extract never
entered the loop, so the assertion held having verified nothing. It also only
scraped install.sh's FILES array despite claiming both bootstraps, so a
Windows-only fetch absent from manifest.sha256 went unnoticed.

Now scrape BOTH bootstraps -- install.sh's FILES and install.ps1's $Files,
using the same awk/sed extractions gen-manifest.sh itself uses -- and fail
loudly when either extract is empty (a broken scrape means the parser went
stale, not that coverage is complete), before the missing-count loop runs.

Bugbot, client#709.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@LukasWodka

Copy link
Copy Markdown
Contributor Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit b9176c1. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant