refactor: extract setup-r-deps and stage-gdrive-auth composite actions - #24
Conversation
R-CMD-check.yaml was a 375-line monolith: the dependency setup could not be reused without also running R CMD check, and the gdrive staging block was duplicated verbatim in test-coverage.yaml with the two copies already drifted. Extract both into composite actions and have the workflow call them; the workflow drops to ~180 lines. Also align the older composite actions with the choices the workflows settled on: install-spatial-deps drops the ubuntugis PPA (ABI-incompatible with Posit's noble binaries, which the workflows have refused for months) and gains the macOS PROJ_DATA configuration; install-Require's default GitTag moves off the non-existent master branch. The retry fallback now mirrors the primary step's dependencies and extra-packages rather than silently resolving a narrower set. timeout-minutes is unsupported in composite steps (actions/runner#1979), so the apt steps use GNU timeout and the dependency retry uses nick-fields/retry's own timeout_minutes. See NEWS.md. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The migration dropped it, so the matrix produces two jobs both called `ubuntu-latest (release)`. You cannot tell from the checks list whether the no-suggests leg passed, branch protection cannot name it as a required check, and if it stopped being generated nobody would notice. Appended only when true, so the other job names stay stable for anyone who has already named them in branch protection. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Pushed one more fix onto this branch (0dd2cc0): the The migration dropped it, so the matrix currently produces two jobs both called name: ${{ matrix.config.os }} (${{ matrix.config.r }}${{ matrix.config.nosuggests && ', nosuggests' || '' }})Appended only when true, so the other job names stay stable for anyone who has already named them in branch protection. |
|
Tracking the downstream half of this in PredictiveEcology/SpaDES.tools#123 — SpaDES.tools pins these workflows to c2b2459, so it won't pick up the Noted there, and worth repeating here: |
|
@achubaty Thanks — this is the right decomposition. Before it lands, here's where I want the whole ecosystem to end up, since this PR is the first time a reusable workflow here depends on a sibling action and that changes the pinning story. Pinning:
|
| repo | current | → |
|---|---|---|
SpaDES.tools |
workflows @ c2b2459 (all 4) |
@main |
LandR |
install-spatial-deps@v0.1 |
@main |
map |
install-spatial-deps@v0.2 |
@main |
SpaDES.shiny |
install-spatial-deps@v0.2 |
@main |
The three on v0.1/v0.2 are still getting the ubuntugis PPA today.
Also in this repo: the usage examples in all four workflow headers still say @v0.4/@v0.5. Anyone following our own docs onboards straight into both the PPA and the pre-e74180e no-suggests bug. Please flip those to @main in this PR.
Re: your v0.5 warning — correct (it's 62d556f, and e74180e is the very next commit), but I checked and nothing in the org actually pins v0.5, so it's a latent trap rather than a live bug. Once we're all on @main I'd rather delete the stale tags than maintain them.
Service accounts are no longer accepted anywhere in CI. A service account authenticates cleanly and then cannot upload to user-owned Drive folders, so upload-backed tests report zero coverage while the job stays green. Staging one is now a hard failure -- a failed step is the only signal that cannot be mistaken for "no credential configured". GOOGLEDRIVE_AUTH is no longer exported to the check/coverage steps; GDRIVE_OAUTH_TOKEN is the contract, which is what reproducible and SpaDES.project@development already read. Adds a self-test workflow, the first CI this repo has ever had. The reusable workflows are workflow_call:-only, so nothing ran on a pull request and every merge reached the repos pinned at @main unverified. It references the composite actions by LOCAL path, which resolves to the PR's checkout rather than main, so a new action can be verified before it exists on main. Covers setup-r-deps on all three runners (loading terra and reprojecting, since gdal-config proves only that headers exist, not that the ABI matches), stage-gdrive-auth across all five credential shapes including the rejected service account, and install-spatial-deps independently -- it has its own consumers. Also routes the setup-r-deps retry fallback's inputs through env: instead of interpolating them into the Rscript command, matching stage-gdrive-auth, and points the four usage examples at @main. v0.4 and v0.5 both carry the ubuntugis PPA and predate e74180e, so the docs were onboarding people into two known bugs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa
|
Pushed a847762 onto this branch, folding in the review points above plus two decisions taken since. What is in itService accounts are gone from CI. Not deprecated — rejected, and the rejection fails the step. A service account authenticates cleanly and then cannot upload to user-owned Drive folders, so upload-backed tests report zero coverage while the job stays green; under A self-test workflow — the first CI this repo has ever had. The reusable workflows are It covers The Merge orderMoot now. The self-test verifies the actions from the PR's own checkout, so this can merge on review. Next, in order
Open question, not blocking: |
…ng CLI
Two bugs the new self-test caught on its first run.
stage-gdrive-auth was broken for every input, not just some. Its `credential`
and `present` descriptions cited `${{ secrets.GOOGLEDRIVE_AUTH }}` as prose
examples, but GitHub evaluates expressions inside `description:` as well, and
`secrets` is not a valid context in a composite action. One such reference
fails the entire manifest to load with "Unrecognized named-value", so the action
never ran at all. Added a note so it is not reintroduced.
The version assertion invoked `proj --version`, but the proj CLI ships in
proj-bin, which libproj-dev does not pull in -- so the check that exists to make
a partial install fail loudly instead failed every complete install on Linux,
and is absent on the macOS runner too. Uses `pkg-config --modversion proj`
instead, whose .pc file ships with the -dev package on both platforms, and adds
pkg-config to the apt list rather than assuming the runner has it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa
The gdrive staging block was ~55 lines duplicated between R-CMD-check.yaml and test-coverage.yaml, and this PR moved only the first copy to the composite action. Dropping service-account support then had to be written twice, once in the action and once in the inline twin -- which is the drift this extraction exists to prevent, reintroduced in the same change that fixed it. test-coverage.yaml: 315 -> 250 lines. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa
Its macOS branch is character-for-character the same `brew install pkg-config gdal` plus PROJ_DATA pair that the setup-r-deps macOS leg already runs. That brew install pulls ~3GB of bottles (llvm 1.9GB, aws-sdk-cpp 640MB, boost, gcc) and takes ~200s; macOS runners bill at 10x, so the duplicate cost more than every Linux job in the run combined while proving nothing new. Wall clock is unchanged -- the run is still bounded by setup-r-deps (macOS) at ~283s -- so this is a billing fix, not a latency one. Cutting that 283s means not installing brew gdal at all, which is a behavioural change worth testing separately. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa
|
Correction to my earlier comment: step 1 said "merge this, then watch NetLogoR" as though the run starts by itself. It does not. None of the five Actions at a mutable ref are fetched at job-execution time, so a re-run picks up Also worth correcting: I listed LandR, map and SpaDES.shiny as "still getting the ubuntugis PPA", which is true but overstates it. The libgdal34/37 ABI mismatch only bites when Posit's prebuilt binaries meet the PPA's libgdal37. LandR and map both set Follow-up in #25: |
#25) The two actions carried two copies of the same apt/brew/PROJ_DATA logic. That duplication is exactly how the ubuntugis PPA survived in install-spatial-deps for months after the reusable workflows had rejected it, and it meant the PPA removal in #24 had to be written twice. One implementation now. Referenced as PredictiveEcology/actions/install-spatial-deps@main rather than ./install-spatial-deps: inside a composite action a relative `uses:` resolves against GITHUB_WORKSPACE -- the CALLING repository's checkout -- so the relative form works only when the caller happens to be this repo and breaks every real consumer. The self-test cannot catch that, since there the workspace is this repo and both resolutions coincide. This could not go in #24: pre-merge, @main was still the copy with the PPA and without the version assertion, so the self-test would have exercised stale spatial code and likely failed on the ABI mismatch #24 removes. setup-r-deps: 250 -> 201 lines. Ports setup-r-deps' stronger `timeout --kill-after=60s` into the surviving copy so nothing is lost, and drops the now-obsolete "verified identical by inspection" caveat in the self-test. Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Blocks the migration of most remaining packages onto these workflows. A survey of the 17 repos still carrying their own R-CMD-check.yaml found per-package env this workflow cannot enumerate and had no way to accept: OMP_THREAD_LIMIT (quickPlot), R_KEEP_PKG_SOURCE (climateData, fpCompare), R_PKG_INSTALL_RETRIES (SpaDES.shiny), _R_CHECK_THINGS_IN_OTHER_DIRS_ (Require). Without this input those packages could not migrate without silently dropping settings, and "silently dropped" is the failure mode this repo keeps having to dig out of. Newline-separated KEY=VALUE, blank lines and #-comments ignored, applied before setup-r-deps so it reaches dependency resolution and not just the check. A line that is not KEY=VALUE fails the step rather than being skipped. Also refreshes the GOOGLEDRIVE_AUTH secret description, which still advertised service-account keys as accepted after #24 made them a hard failure. Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
test-coverage does not call setup-r-deps -- it still carries its own inline pandoc and apt steps -- so the inputs are wired directly onto those steps rather than passed through. Both default to true; nothing changes for existing callers. Smaller payoff than on R-CMD-check: this job is ubuntu-only, so it saves an apt install rather than the ~3GB macOS Homebrew pull. Also worth noting the comment above its spatial block is now stale: it warns that install-spatial-deps uses the ubuntugis PPA, which #24 removed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa
* feat(R-CMD-check): expose system-deps and pandoc to callers setup-r-deps has had both switches since it was extracted, but the reusable workflow neither exposed nor passed them, so every caller installed the geospatial stack and pandoc whether or not it could use them. Six packages in the org declare no spatial dependency at all -- fpCompare, Require, pedev, peutils, SpaDES.install, SpaDES.addins -- and each was pulling ~3GB of Homebrew (llvm, gcc, boost, aws-sdk-cpp, ~200s) on its macOS leg at the 10x macOS billing rate, plus an apt install on every Linux leg, for libraries they never load. Both default to true, so no existing caller changes behaviour. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa * feat(test-coverage): honour system-deps and pandoc too test-coverage does not call setup-r-deps -- it still carries its own inline pandoc and apt steps -- so the inputs are wired directly onto those steps rather than passed through. Both default to true; nothing changes for existing callers. Smaller payoff than on R-CMD-check: this job is ubuntu-only, so it saves an apt install rather than the ~3GB macOS Homebrew pull. Also worth noting the comment above its spatial block is now stale: it warns that install-spatial-deps uses the ubuntugis PPA, which #24 removed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
What
Extracts two composite actions out of the
R-CMD-checkreusable workflow, andbrings the older composite actions in line with the choices those workflows
settled on.
R-CMD-check.yaml: 375 → ~180 lines.setup-r-depssetup-r, and the dependency install with its retry fallbackstage-gdrive-authGOOGLEDRIVE_AUTHonto the runner and reports the credential type as typed outputsWhy
The dependency setup was not reusable without also running
R CMD check. Abenchmark job, an ad-hoc matrix leg, or a coverage run with different flags had
to copy ~90 lines to get "install this package's deps the PredictiveEcology
way". That is the whole point of the extraction.
The copies had already drifted. The gdrive staging block was ~55 lines
duplicated verbatim between
R-CMD-check.yamlandtest-coverage.yaml, and thediagnostic that exists specifically to make a dropped secret visible was less
informative in one of the two. The apt block exists in four copies in two
camps, two of which pass
--fix-missing— which skips unfetchable packages andstill exits 0.
Fixes carried along
pak's own
dependencies = TRUE(narrower than"all") and droppedextra-packagesentirely, so a caller pinning a dependency viaextra-packageswas quietly checked against the CRAN version instead. Becausethe primary step is
continue-on-error: true, nothing in the UI showed thishad happened. It now mirrors the primary step and emits a warning when it
fires.
cache-versionwas unreachable. The restore-key carries no discriminatorfor the dependency set, so a leg can inherit a sibling leg's library — and
clearing a poisoned cache (e.g. a
terrabuilt against the wronglibgdal)needed a PR to this repo. Now an input.
install-spatial-depsadded theubuntugis-unstablePPA that thereusable workflows have explicitly refused for months, because it installs
libgdal37 against Posit's libgdal34 binaries. The org was shipping two
contradictory answers to the same question, and
SpaDES.toolsconsumes both.Removed; also gains the macOS
PROJ_DATAconfiguration the workflows had andthis action lacked, plus a version assertion so a partial install fails loudly.
install-Requiredefaulted toGitTag: master, a branch that no longerexists on
PredictiveEcology/Require. Every caller happens to override it, sothe default was dead code that would hard-fail if used. Now
development.On timeouts
timeout-minutesis not supported for steps inside a composite action(actions/runner#1979), and it
is not honoured on the step that calls one either — worth knowing, because
that is the natural thing to reach for. Three alternatives, all used here:
timeoutinside therun:block — the apt steps. Gives aper-command cap rather than one cap for the whole step, which is strictly
better than the
timeout-minutes: 15it replaces. Linux/macOS only; Windowsrunners have no GNU
timeout.uses:other actions, so
nick-fields/retry'stimeout_minutesworks. Used for thedependency retry.
timeout-minuteson the calling job — a coarse backstop; kills the wholejob, not the step.
R-CMD-check.yamlreferencesPredictiveEcology/actions/setup-r-deps@mainand.../stage-gdrive-auth@main.uses:cannot take an expression, so it cannot bepointed at the PR branch. CI on this PR will fail until the actions exist on
main— that is expected, not a defect in the change. Options:main; oruses:lines at@refactor/extract-composite-actions, verify green, and revert to@maininthe same PR before merging.
I have not done the second automatically, since it leaves a footgun if the
revert is forgotten.
Not in scope
test-coverage.yaml,test-downstream.yamlandpkgdown.yamlstill carrytheir own copies of the setup blocks. They should move to
setup-r-depstoo,but doing all four at once makes this unreviewable. Happy to follow up.
🤖 Generated with Claude Code