Skip to content

ci: extend CI coverage (CodeQL, PR validation, dependency review) to spark4.1 - #2532

Merged
Rana Singh (ranadeepsingh) merged 1 commit into
masterfrom
openssf/improve-sast-coverage
Aug 13, 2026
Merged

ci: extend CI coverage (CodeQL, PR validation, dependency review) to spark4.1#2532
Rana Singh (ranadeepsingh) merged 1 commit into
masterfrom
openssf/improve-sast-coverage

Conversation

@BrendanWalsh

@BrendanWalsh Brendan Walsh (BrendanWalsh) commented Mar 31, 2026

Copy link
Copy Markdown
Collaborator

Why

Three workflows were gated to master only, so the actively published spark4.1 release branch was missing checks that master gets on every PR. Release branches take cherry-picked fixes and occasional direct commits — none of it was being scanned or validated.

PR targeting master          PR targeting spark4.1  (before)
---------------------        ------------------------------
CodeQL            ✅          CodeQL             ✗
PR Validation     ✅          PR Validation      ✅ (see note)
Dependency Review ✅          Dependency Review  ✗

What changed

Four lines across three files.

Workflow What it does Was Now
codeql.yml SAST / security scanning master master, spark4.1
pr-validation.yml Python style, Scala style, compile master master, spark4.1
dependency-review.yml Supply-chain vulnerability review master master, spark4.1

on.<event>.branches matches the pull request's base branch, so adding spark4.1 is precisely what makes these run for PRs targeting it.

The pr-validation case is a latent regression, not just a gap

spark4.1 already carries branches: [ "master", "spark4.1" ], but master still had [ "master" ]:

Branch pr-validation.yml filter
spark4.1 [ "master", "spark4.1" ]
master [ "master" ]

That drift is a trap: the next time master is synced into spark4.1, master's narrower filter overwrites it and silently disables style + compile validation on spark4.1 PRs. Aligning master removes it.

Deliberately not changed

Scope is kept to where it pays off — scanning branches nobody ships costs CI minutes and produces alerts nobody acts on.

Item Why not
spark3.5 master now tracks Spark 3.5, so that code is already covered via master
spark4.0 Not actively published
spark3.1spark3.4 Unmaintained
check-dead-links.yml Scans the deployed website's sitemap (built from master); it inspects nothing from the PR, so it would add CI time on release-branch PRs for no signal
scorecards.yml OpenSSF Scorecard is designed to assess the default branch
website-deploy.yml Already runs pull_request on all branches; deployment should stay master-only

One note on rollout

For pull_request events GitHub reads the workflow file from the PR's head branch. Since contributors branch from the branch they target, these filters take full effect on spark4.1 once this change is synced into that branch. The change is still correct and necessary on master — it's the prerequisite, and it's what stops the pr-validation regression above.

Checklist

  • All three workflows parse cleanly; verified no duplicate YAML keys
  • spark4.1 confirmed to exist on the remote
  • Compared every workflow's triggers on master vs spark4.1 to find the full gap set
  • Net diff is 4 lines; no change to any job's steps, permissions, or secrets
  • Rebased onto current master

Copilot AI review requested due to automatic review settings March 31, 2026 04:46
@github-actions

Copy link
Copy Markdown

Hey Brendan Walsh (@BrendanWalsh) 👋!
Thank you so much for contributing to our repository 🙌.
Someone from SynapseML Team will be reviewing this pull request soon.

We use semantic commit messages to streamline the release process.
Before your pull request can be merged, you should make sure your first commit and PR title start with a semantic prefix.
This helps us to create release messages and credit you for your hard work!

Examples of commit messages with semantic prefixes:

  • fix: Fix LightGBM crashes with empty partitions
  • feat: Make HTTP on Spark back-offs configurable
  • docs: Update Spark Serving usage
  • build: Add codecov support
  • perf: improve LightGBM memory usage
  • refactor: make python code generation rely on classes
  • style: Remove nulls from CNTKModel
  • test: Add test coverage for CNTKModel

To test your commit locally, please follow our guild on building from source.
Check out the developer guide for additional guidance on testing your change.

@github-actions

github-actions Bot commented Mar 31, 2026

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@ranadeepsingh Rana Singh (ranadeepsingh) changed the title ci: expand CodeQL SAST coverage to all active branches ci: extend CodeQL SAST coverage to the spark4.1 release branch Aug 13, 2026
@ranadeepsingh

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Three workflows were gated to `master` only, so the actively published
spark4.1 release branch was missing checks that master gets on every PR.

| Workflow | Was | Now |
| --- | --- | --- |
| codeql.yml | master | master, spark4.1 |
| pr-validation.yml | master | master, spark4.1 |
| dependency-review.yml | master | master, spark4.1 |

The `on.<event>.branches` filter matches the pull request's BASE branch, so
adding `spark4.1` is what makes these run for PRs targeting that branch.

pr-validation.yml is the notable one: the spark4.1 branch already carries
`branches: [ "master", "spark4.1" ]`, but master still had `[ "master" ]`.
That drift is a latent regression — the next time master is synced into
spark4.1, master's narrower filter would overwrite it and silently disable
style and compile validation on spark4.1 pull requests. Aligning master
removes that trap.

Deliberately not changed:
- spark3.5 / spark4.0 / spark3.x: master now tracks Spark 3.5, spark4.0 is
  not actively published, and the older branches are unmaintained.
- check-dead-links.yml: it scans the deployed website's sitemap, which is
  built from master, so it would add CI time on release-branch PRs without
  inspecting anything from the pull request.
- scorecards.yml: OpenSSF Scorecard is designed to assess the default branch.
- website-deploy.yml: already runs on `pull_request` for all branches, and
  deployment should stay master-only.

Note: for `pull_request` events GitHub reads the workflow file from the PR's
head branch, so these filters take full effect on spark4.1 once this change
is synced into that branch. It is still correct and necessary on master.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ranadeepsingh Rana Singh (ranadeepsingh) changed the title ci: extend CodeQL SAST coverage to the spark4.1 release branch ci: extend CI coverage (CodeQL, PR validation, dependency review) to spark4.1 Aug 13, 2026
@ranadeepsingh

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@codecov-commenter

Codecov Comments Bot (codecov-commenter) commented Aug 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.91%. Comparing base (ffe123a) to head (8283099).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2532      +/-   ##
==========================================
- Coverage   87.11%   84.91%   -2.21%     
==========================================
  Files         338      338              
  Lines       18858    18858              
  Branches     1777     1777              
==========================================
- Hits        16429    16013     -416     
- Misses       2429     2845     +416     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ranadeepsingh
Rana Singh (ranadeepsingh) merged commit 7be2767 into master Aug 13, 2026
72 checks passed
@ranadeepsingh
Rana Singh (ranadeepsingh) deleted the openssf/improve-sast-coverage branch August 13, 2026 01:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants