Repository navigation
feat(e2e): add supportability-matrix CI workflow and supporting fixes - #330
Merged
Merged
Conversation
Adds .github/workflows/supportability-matrix.yml, which tests tf-migrate's
v4->v5 config migration against every provider version in the known
supportability matrix (5.19.0, 5.19.1, 5.20.0, 5.21.1, 5.22.0, 5.23.0),
triggered weekly and via workflow_dispatch (optionally targeting a single
version). Legs run sequentially against one shared, persistent v4 resource
base, installing each target provider version from the public registry.
Includes the fixes this workflow depends on to produce accurate results:
- internal/e2e-runner/runner.go, version_suffix.go: fix a bug where the
harness's obsolete-state-cleanup step deleted state entries by type name
alone, including ones with a moved {} counterpart - causing
zero_trust_access_policy (and any similarly-shaped resource) to be
destroyed and recreated on every run instead of correctly reusing existing
state. Scoped the cleanup to only entries without a moved {} block.
- e2e/drift-exemptions/{argo,tiered_cache,zero_trust_access_policy,
zero_trust_tunnel_cloudflared_route,zone_setting}.yaml: scope previously
unscoped "allow_resource_creation: true" exemptions to their intended
resource_types/resource_name_patterns. These were silently masking the
state-cleanup bug above (and similar churn) across every resource type
they matched.
- e2e/drift-exemptions/zero_trust_organization.yaml (new): scoped exemption
for the v5.23.0 mfa_required_for_all_apps normalization one-time diff.
- e2e/drift-exemptions/zero_trust_access_group.yaml (new): scoped exemption
for one-time condition-explosion list-reordering drift on first apply.
- integration/v4_to_v5/testdata/zero_trust_access_group/input/
zero_trust_access_group_e2e.tf (new): e2e-safe variant omitting only the
gsuite_list_email pattern, which requires a real Google Workspace
(googleapps) identity provider that test environments may not have
configured. All other 34 patterns in the full testdata are confirmed to
work against the live API, including ones using fake identity_provider_id
values for azure/okta selectors.
- .gitignore: ignore version-isolated local directories
(e2e/tf/v4-*/, e2e/migrated-v4_to_v5-*/, e2e/tmp-*/) used by
--version-suffix runs.
GitHub Actions rejects the matrix context in a job-level if: condition
("Unrecognized named-value: 'matrix'"), which caused the first CI run
on this branch to fail with zero jobs created before any test logic ran.
Moved the same condition to the step-level if on the actual test-execution
step, where the matrix context is valid. Non-matching legs now still
create a job (cheap checkout/setup steps run) but skip the real e2e run.
TEMPORARY commit — adds a push trigger scoped to this branch only, so the workflow_dispatch-only workflow can get one real validation run before merge (workflow_dispatch requires the workflow to already exist on the default branch, which it doesn't yet). Will be reverted before merge.
…edential-check pattern
The push trigger scoped to this PR branch was only added to get one real CI run before merge, since workflow_dispatch can't trigger a workflow that doesn't yet exist on the default branch. It's served its purpose — the workflow ran successfully end-to-end for all 6 matrix legs (run 37835491990) after the matrix-context-in-job-if fix. Final triggers are workflow_dispatch (optionally targeting a single version) and the weekly schedule only.
musa-cf
approved these changes
Oct 9, 2026
Both are superseded by supportability-matrix.yml, which covers every
version e2e-tests.yml tested (and five more) by installing from the
registry instead of building from source. e2e-tests-next.yml (manual-only,
builds the provider from its next branch tip) is removed too — accepting
the loss of next-branch regression testing for now.
Also fixes now-stale references to the deleted e2e-tests.yml:
- Go doc comments in cmd/e2e-runner/main.go and internal/e2e-runner/
{init,runner,version_suffix}.go describing the shared/unsuffixed state
key and directories no longer name a specific deleted workflow as the
example consumer of that shared slot.
- e2e/README.md's CI/CD section now points at supportability-matrix.yml
and describes its actual triggers (weekly schedule + manual dispatch)
instead of the removed push-to-main workflow.
mgirouard
approved these changes
Oct 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
.github/workflows/supportability-matrix.yml, which tests tf-migrate's v4→v5 config migration against every provider version in the known supportability matrix (5.19.0, 5.19.1, 5.20.0, 5.21.1, 5.22.0, 5.23.0), triggered weekly and viaworkflow_dispatch(optionally targeting a single version). Legs run strictly sequentially against one shared, persistent v4 resource base, installing each target provider version fresh from the public Terraform Registry (no local provider build).This does not replace or trigger on push to
mainyet —e2e-tests.ymlstays in place for now. That migration, plus README/provider-migration-guide updates, will follow in a separate change.Supporting fixes this workflow depends on
internal/e2e-runner/runner.go,version_suffix.go: fixes a bug where the harness's obsolete-state-cleanup step deleted state entries by type name alone, including ones with amoved {}counterpart — causingzero_trust_access_policy(and similarly-shaped resources) to be destroyed and recreated on every run instead of reusing existing state. Cleanup is now scoped to only entries without amoved {}block.e2e/drift-exemptions/{argo,tiered_cache,zero_trust_access_policy,zero_trust_tunnel_cloudflared_route,zone_setting}.yaml: scope previously unscopedallow_resource_creation: trueexemptions to their intendedresource_types/resource_name_patterns. These were silently masking the state-cleanup bug above (and similar churn).e2e/drift-exemptions/zero_trust_organization.yaml(new): scoped exemption for the v5.23.0mfa_required_for_all_appsnormalization one-time diff.e2e/drift-exemptions/zero_trust_access_group.yaml(new): scoped exemption for one-time condition-explosion list-reordering drift on first apply.integration/v4_to_v5/testdata/zero_trust_access_group/input/zero_trust_access_group_e2e.tf(new): e2e-safe variant omitting only thegsuite_list_emailpattern, which requires a real Google Workspace (googleapps) identity provider not available in test environments. All other 34 patterns are confirmed working against the live API..gitignore: ignore version-isolated local directories (e2e/tf/v4-*/,e2e/migrated-v4_to_v5-*/,e2e/tmp-*/) used by--version-suffixruns.Testing
go build ./...— cleango test ./internal/...— all passmake lint-testdata— pass