ci(): check PYTHON_VERSIONS against DuckDB wheels and Lambda runtimes #29
Merged
Merged
Conversation
f708077 to
b031f67
Compare
This was referenced Jun 21, 2026
bengeois
reviewed
Jun 22, 2026
bengeois
left a comment
Owner
There was a problem hiding this comment.
Thanks @Gudsfile 🙏🏼
Open questions
- I'd keep it standalone. The triggers are different (no need to run on workflow_dispatch) and it keeps build-layer.yml focused on the build. No strong reason to merge them.
- Not critical given the script's size, but it would be worth adding tests for fetch_duckdb_pythons (with a mocked HTTP response) and read_python_versions (with a fixture YAML) to catch silent regressions if the PyPI response format ever changes.
Suggestion
- Also, on the bump PRs created by #25, since the branch belongs to the repo (not a fork), the bump-duckdb.yml workflow could theoretically compute the correct PYTHON_VERSIONS at bump time, by running the same intersection logic and write it directly into the bump commit, rather than failing the check after the fact. This would make the whole pipeline fully hands-off.
Is that something you'd consider tackling, either in this PR or as a follow-up?
b031f67 to
fe2f611
Compare
Collaborator
Author
|
Standalone workflow Tests
Both wired into CI in Suggestion (bump-time computation) |
bengeois
reviewed
Jun 29, 2026
Add a pull_request workflow that fails if PYTHON_VERSIONS in build-layer.yml diverges from the intersection of DuckDB manylinux wheels (from PyPI) and supported Lambda runtimes (.github/.lambda-python-runtimes). Catches both missing versions (DuckDB added a Python not in the matrix) and stale versions (DuckDB dropped a Python still in the matrix or a Lambda runtime was removed). Lambda runtimes are hardcoded in .github/.lambda-python-runtimes and must be updated manually when AWS announces a new runtime. Print an error when the DuckDB version in .duckdb-version does not exist on PyPI instead of crashing with an unhandled HTTPError.
fe2f611 to
561f1c6
Compare
bengeois
approved these changes
Jun 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follows up on #25.
Adds a
pull_requestworkflow that fails ifPYTHON_VERSIONSinbuild-layer.ymldiverges from the intersection of DuckDB manylinux wheels (from PyPI) and supported Lambda runtimes (hardcoded).Why
The build matrix only validates versions that are already listed, it cannot detect when DuckDB ships wheels for a new Python version absent from the matrix. This check fills that gap, and also catches stale versions (DuckDB dropped support, or Lambda runtime was removed).
Allows you to safely use auto-merge without requiring a review (#28).
What does it look like
Tests
To avoid making noise on this PR, I tested it in another PR: #31
Open question
build-layer.yml. Open to merging it intobuild-layer.ymlonpull_requestevents if preferred.check_python_versions.pyand a CI to run them?Out of scope
A follow-up could automate this by scrapping https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html (or any better idea).