Skip to content

ci(env): one CLIENT_ENV vocabulary, four languages, now checked (backend#1729 sweep 5) - #706

Merged
LukasWodka merged 3 commits into
developfrom
ci/1729-env-vocabulary-agreement
Aug 13, 2026
Merged

ci(env): one CLIENT_ENV vocabulary, four languages, now checked (backend#1729 sweep 5)#706
LukasWodka merged 3 commits into
developfrom
ci/1729-env-vocabulary-agreement

Conversation

@LukasWodka

@LukasWodka LukasWodka commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Sweep 5 of the inert-verification epic. The epic states the class as:

"A verification written in the same vocabulary as the thing it verifies
cannot detect a vocabulary error."

and its evidence was 366 chart tests covering dev/stg/prod/unset/unknown
while NOT ONE set staging -- the alias the chart's own docs recommend.

That measured gap is now closed (staging has 9 cases, development 2,
production 4). What is NOT closed is the structure underneath it: the same
three alias->canonical mappings are declared FOUR times, in four
languages, and nothing compares them.

1 client/templates/_helpers.tpl $aliases := dict ... Go template
2 client/values.schema.json the CLIENT_ENV enum JSON Schema
3 scripts/lib/common.sh tb_client_env() bash case
4 scripts/install-k8s.ps1 Get-TraceblocClientEnv PowerShell switch

Two of the four have already drifted, separately, and been repaired
separately: backend#1723 fixed the chart, backend#1745 fixed the bash
installer -- whose own comment records the cost, "a raw staging fell
through to the prod branch, so verify_credentials() checked staging
credentials against the production backend and reported them invalid".

So adding a seventh spelling to the template leaves both installers
silently not reducing it, which is #1745 reintroduced in a repo that has
already paid for it once.

THE GUARD DERIVES, IT DOES NOT RESTATE. It parses all four declarations
and compares them to each other; it holds no copy of the vocabulary,
because a fifth hand-written list is the defect rather than the fix (the
lesson of backend#1780 and backend#1828, where hand-copied declarations
each claimed the others kept them honest and nothing crossed the
boundary). It also asserts every accepted spelling is exercised by at
least one helm-unittest case -- the specific thing #1729 measured.

ARMED WHILE GREEN, deliberately: all four agree today and all six
spellings are tested, so this imports no backlog. Arming a red check
trains people to skip the tier -- the same reasoning as .github#235 and
the opposite of what happened when a fleet-wide copies: bump reddened the
org audit for 2h20m on 2026-08-12.

Mutations, all five behaving correctly:
template gains a 4th alias, installers unchanged 3 findings
bash reducer drops staging (the #1745 shape) 1 finding
PowerShell maps staging -> dev 1 finding
schema accepts a spelling no reducer maps 2 findings
a parser goes stale (dict renamed) EXIT 2, fail-closed

That last one matters most: zero parsed pairs compares equal to zero
parsed pairs, so a stale parser would report agreement between four
declarations it never read. It exits 2 with a diagnostic instead.

Wired into SHELLCHECK_FILES and make check beside its sibling, and into
helm-ci with path filters covering all four declarations -- including
scripts/lib/common.sh and scripts/install-k8s.ps1, so a change to either
installer runs it.

make check green; chart-env-vocabulary 28/28; gen-manifest --check clean.

Co-Authored-By: Claude Opus 5 noreply@anthropic.com


Note

Low Risk
CI and local test wiring only; no runtime chart or installer behavior changes, with fail-closed exit codes for parse/tooling errors.

Overview
Adds scripts/tests/env-vocabulary-agreement.sh, a read-only guard that parses (does not duplicate) the CLIENT_ENV alias vocabulary from the Helm helper dict, values.schema.json enum, tb_client_env() in common.sh, and Get-TraceblocClientEnv in install-k8s.ps1, then fails if those mappings or the schema’s accepted spellings diverge. It also requires each accepted spelling to appear in at least one helm-unittest under client/tests/, closing the structural gap behind backend#1729.

Wiring: the script runs from make helm-vocab / make check, is included in SHELLCHECK_FILES, gets a new helm-ci lint step, and helm-ci path filters now cover common.sh and install-k8s.ps1 so installer edits re-run the check. make setup now expects python3 on PATH (used to read the schema enum).

Reviewed by Cursor Bugbot for commit bcbfc0e. Bugbot is set up for automated code reviews on this repo. Configure here.

…end#1729 sweep 5)

Sweep 5 of the inert-verification epic. The epic states the class as:

  "A verification written in the same vocabulary as the thing it verifies
   cannot detect a vocabulary error."

and its evidence was 366 chart tests covering dev/stg/prod/unset/unknown
while NOT ONE set `staging` -- the alias the chart's own docs recommend.

That measured gap is now closed (staging has 9 cases, development 2,
production 4). What is NOT closed is the structure underneath it: the same
three alias->canonical mappings are declared FOUR times, in four
languages, and nothing compares them.

  1  client/templates/_helpers.tpl   $aliases := dict ...      Go template
  2  client/values.schema.json       the CLIENT_ENV enum       JSON Schema
  3  scripts/lib/common.sh           tb_client_env()           bash case
  4  scripts/install-k8s.ps1         Get-TraceblocClientEnv    PowerShell switch

Two of the four have already drifted, separately, and been repaired
separately: backend#1723 fixed the chart, backend#1745 fixed the bash
installer -- whose own comment records the cost, "a raw `staging` fell
through to the prod branch, so verify_credentials() checked staging
credentials against the production backend and reported them invalid".

So adding a seventh spelling to the template leaves both installers
silently not reducing it, which is #1745 reintroduced in a repo that has
already paid for it once.

THE GUARD DERIVES, IT DOES NOT RESTATE. It parses all four declarations
and compares them to each other; it holds no copy of the vocabulary,
because a fifth hand-written list is the defect rather than the fix (the
lesson of backend#1780 and backend#1828, where hand-copied declarations
each claimed the others kept them honest and nothing crossed the
boundary). It also asserts every accepted spelling is exercised by at
least one helm-unittest case -- the specific thing #1729 measured.

ARMED WHILE GREEN, deliberately: all four agree today and all six
spellings are tested, so this imports no backlog. Arming a red check
trains people to skip the tier -- the same reasoning as .github#235 and
the opposite of what happened when a fleet-wide copies: bump reddened the
org audit for 2h20m on 2026-08-12.

Mutations, all five behaving correctly:
  template gains a 4th alias, installers unchanged   3 findings
  bash reducer drops staging (the #1745 shape)       1 finding
  PowerShell maps staging -> dev                     1 finding
  schema accepts a spelling no reducer maps          2 findings
  a parser goes stale (dict renamed)                 EXIT 2, fail-closed

That last one matters most: zero parsed pairs compares equal to zero
parsed pairs, so a stale parser would report agreement between four
declarations it never read. It exits 2 with a diagnostic instead.

Wired into SHELLCHECK_FILES and `make check` beside its sibling, and into
helm-ci with path filters covering all four declarations -- including
scripts/lib/common.sh and scripts/install-k8s.ps1, so a change to either
installer runs it.

make check green; chart-env-vocabulary 28/28; gen-manifest --check clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@LukasWodka LukasWodka self-assigned this Aug 13, 2026
@LukasWodka
LukasWodka requested a review from saadqbal August 13, 2026 12:39
Comment thread scripts/tests/env-vocabulary-agreement.sh
…ary gap

env-vocabulary-agreement.sh reads the CLIENT_ENV enum out of values.schema.json
with python3, but `make setup` never installed (or even checked for) python3, so
on the pre-push `make check` path a missing interpreter failed closed with the
false diagnosis that the schema has no CLIENT_ENV enum and the vocabulary is no
longer closed. The Go/bash/PowerShell reducers are jq-free by rule, so python3 is
the JSON parser here and was the one unguarded dependency.

Preflight `command -v python3` before the enum is read, and at the call site
branch on the helper's exit status: 3 (its sys.exit for a genuinely-absent enum)
still reports the real closed-vocabulary finding, while any other non-zero (a
python3 that fails to run, malformed JSON) reports a distinct tooling/parse
error. Add python3 to `make setup`'s prereq check so the gap is caught up front.

Bugbot, client#706.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@LukasWodka

Copy link
Copy Markdown
Contributor Author

bugbot run

Comment thread Makefile
The Makefile's SHELLCHECK_FILES already lists env-vocabulary-agreement.sh,
but installer-tests.yaml's static job shellchecked only through
chart-env-vocabulary.sh, so CI never linted the new guard. Append it to
both the error-gate and warning-advisory invocations so the workflow list
matches the Makefile again (same files, same order).

Bugbot, client#706.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@LukasWodka

Copy link
Copy Markdown
Contributor Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit bcbfc0e. Configure here.

@LukasWodka
LukasWodka merged commit 3e97d10 into develop Aug 13, 2026
47 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant