ci: extend CI coverage (CodeQL, PR validation, dependency review) to spark4.1 - #2532
Conversation
|
Hey Brendan Walsh (@BrendanWalsh) 👋! We use semantic commit messages to streamline the release process. Examples of commit messages with semantic prefixes:
To test your commit locally, please follow our guild on building from source. |
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
ad54a3c to
02ca210
Compare
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Three workflows were gated to `master` only, so the actively published spark4.1 release branch was missing checks that master gets on every PR. | Workflow | Was | Now | | --- | --- | --- | | codeql.yml | master | master, spark4.1 | | pr-validation.yml | master | master, spark4.1 | | dependency-review.yml | master | master, spark4.1 | The `on.<event>.branches` filter matches the pull request's BASE branch, so adding `spark4.1` is what makes these run for PRs targeting that branch. pr-validation.yml is the notable one: the spark4.1 branch already carries `branches: [ "master", "spark4.1" ]`, but master still had `[ "master" ]`. That drift is a latent regression — the next time master is synced into spark4.1, master's narrower filter would overwrite it and silently disable style and compile validation on spark4.1 pull requests. Aligning master removes that trap. Deliberately not changed: - spark3.5 / spark4.0 / spark3.x: master now tracks Spark 3.5, spark4.0 is not actively published, and the older branches are unmaintained. - check-dead-links.yml: it scans the deployed website's sitemap, which is built from master, so it would add CI time on release-branch PRs without inspecting anything from the pull request. - scorecards.yml: OpenSSF Scorecard is designed to assess the default branch. - website-deploy.yml: already runs on `pull_request` for all branches, and deployment should stay master-only. Note: for `pull_request` events GitHub reads the workflow file from the PR's head branch, so these filters take full effect on spark4.1 once this change is synced into that branch. It is still correct and necessary on master. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
02ca210 to
8283099
Compare
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #2532 +/- ##
==========================================
- Coverage 87.11% 84.91% -2.21%
==========================================
Files 338 338
Lines 18858 18858
Branches 1777 1777
==========================================
- Hits 16429 16013 -416
- Misses 2429 2845 +416 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Why
Three workflows were gated to
masteronly, so the actively publishedspark4.1release branch was missing checks thatmastergets on every PR. Release branches take cherry-picked fixes and occasional direct commits — none of it was being scanned or validated.What changed
Four lines across three files.
codeql.ymlmastermaster,spark4.1pr-validation.ymlmastermaster,spark4.1dependency-review.ymlmastermaster,spark4.1on.<event>.branchesmatches the pull request's base branch, so addingspark4.1is precisely what makes these run for PRs targeting it.The
pr-validationcase is a latent regression, not just a gapspark4.1already carriesbranches: [ "master", "spark4.1" ], butmasterstill had[ "master" ]:pr-validation.ymlfilterspark4.1[ "master", "spark4.1" ]✅master[ "master" ]❌That drift is a trap: the next time
masteris synced intospark4.1, master's narrower filter overwrites it and silently disables style + compile validation on spark4.1 PRs. Aligningmasterremoves it.Deliberately not changed
Scope is kept to where it pays off — scanning branches nobody ships costs CI minutes and produces alerts nobody acts on.
spark3.5masternow tracks Spark 3.5, so that code is already covered viamasterspark4.0spark3.1–spark3.4check-dead-links.ymlmaster); it inspects nothing from the PR, so it would add CI time on release-branch PRs for no signalscorecards.ymlwebsite-deploy.ymlpull_requeston all branches; deployment should staymaster-onlyOne note on rollout
For
pull_requestevents GitHub reads the workflow file from the PR's head branch. Since contributors branch from the branch they target, these filters take full effect onspark4.1once this change is synced into that branch. The change is still correct and necessary onmaster— it's the prerequisite, and it's what stops thepr-validationregression above.Checklist
spark4.1confirmed to exist on the remotemastervsspark4.1to find the full gap setmaster