acc: Prefix unique test resource names with CI run id for attribution and sweeping#5586
Draft
chrisst wants to merge 3 commits into
Draft
acc: Prefix unique test resource names with CI run id for attribution and sweeping#5586chrisst wants to merge 3 commits into
chrisst wants to merge 3 commits into
Conversation
… and sweeping Acceptance tests name cloud resources with a bare random string ($UNIQUE_NAME, e.g. 'osr5mzrrvzb73juixjoviti24y'). When such resources leak into shared test workspaces, nothing ties them back to a repo or CI run, which made a recent quota-exhaustion incident hard to diagnose and makes targeted cleanup impossible. When running against a cloud env (CLOUD_ENV set) with GITHUB_RUN_ID present, the harness now generates names of the form 'ci-<GITHUB_RUN_ID>-<random>', truncated to the same 26-character length as before so that length-constrained names built from $UNIQUE_NAME (e.g. 'app-$UNIQUE_NAME', exactly 30 chars, the app name limit) keep fitting. The character set stays lowercase alphanumerics plus '-', which tests already use adjacent to $UNIQUE_NAME in every resource type. Local runs and testserver runs are unchanged, and output masking is unaffected because the whole generated name (prefix included) is still replaced with [UNIQUE_NAME]. Also adds tools/sweep_test_resources.py, a small helper that lists (and with --delete removes) warehouses, pipelines and jobs whose names start with a given prefix, to be wired into CI job-end cleanup later. Co-authored-by: Isaac
Co-authored-by: Isaac
Drop the CLOUD_ENV gate: ciUniqueName already no-ops without a valid run id, and prefixing testserver runs in CI exercises the masking path everywhere instead of only on cloud runs. Co-authored-by: Isaac
Collaborator
Integration test reportCommit: 162ca50
57 interesting tests: 30 FAIL, 15 SKIP, 7 KNOWN, 3 BUG, 2 flaky
Top 19 slowest tests (at least 2 minutes):
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Makes acceptance-test resource names attributable to the CI run that created them, and sweepable:
GITHUB_RUN_IDis numeric (any run, cloud or testserver),$UNIQUE_NAMEbecomesci-<run-id>-<random>, truncated to the same 26-char total length as today (≥8 random chars guaranteed; falls back to the unprefixed name otherwise). Length preservation matters:app-$UNIQUE_NAMEsits exactly at the 30-char app-name limit;[UNIQUE_NAME]via whole-string replacement;GITHUB_RUN_IDare byte-for-byte unchanged; in-CI testserver runs now exercise the prefixed-name masking path on every PR;tools/sweep_test_resources.py: given a name prefix and CLI auth env, lists (dry-run) or deletes (--delete) matching warehouses/pipelines/jobs — ready to wire into CI job-end;TestCIUniqueNameunit coverage (missing/short/malformed/too-long run ids).Why
Today leaked test resources carry bare random names (e.g.
osr5mzrrvzb73juixjoviti24y) — nothing ties them to a repo, run, or owner. During the 2026-06-11/12 shared-workspace quota incident (ref ES-1974228), attributing 100+ leaked warehouses to their source took hours of archaeology, and targeted cleanup was impossible. Run-scoped prefixes make leaks attributable at a glance and enable prefix-scoped sweeping.Tests
go build ./...,go vet ./acceptance,TestCIUniqueNamepass; local acceptance tests that assert[UNIQUE_NAME]in outputs (cmd/fs/cp,bundle/run_as/job_default) pass withGITHUB_RUN_IDset, confirming replacement behavior is unchanged.This pull request and its description were written by Isaac.