Skip to content

refactor: extract setup-r-deps and stage-gdrive-auth composite actions - #24

Merged
eliotmcintire merged 6 commits into
mainfrom
refactor/extract-composite-actions
Aug 31, 2026
Merged

refactor: extract setup-r-deps and stage-gdrive-auth composite actions#24
eliotmcintire merged 6 commits into
mainfrom
refactor/extract-composite-actions

Conversation

@achubaty

Copy link
Copy Markdown
Contributor

What

Extracts two composite actions out of the R-CMD-check reusable workflow, and
brings the older composite actions in line with the choices those workflows
settled on.

R-CMD-check.yaml: 375 → ~180 lines.

new action what it is
setup-r-deps pandoc (with retries), geospatial system libraries, setup-r, and the dependency install with its retry fallback
stage-gdrive-auth decodes GOOGLEDRIVE_AUTH onto the runner and reports the credential type as typed outputs

Why

The dependency setup was not reusable without also running R CMD check. A
benchmark job, an ad-hoc matrix leg, or a coverage run with different flags had
to copy ~90 lines to get "install this package's deps the PredictiveEcology
way". That is the whole point of the extraction.

The copies had already drifted. The gdrive staging block was ~55 lines
duplicated verbatim between R-CMD-check.yaml and test-coverage.yaml, and the
diagnostic that exists specifically to make a dropped secret visible was less
informative in one of the two. The apt block exists in four copies in two
camps, two of which pass --fix-missing — which skips unfetchable packages and
still exits 0.

Fixes carried along

  • The retry fallback silently checked a different dependency set. It used
    pak's own dependencies = TRUE (narrower than "all") and dropped
    extra-packages entirely, so a caller pinning a dependency via
    extra-packages was quietly checked against the CRAN version instead. Because
    the primary step is continue-on-error: true, nothing in the UI showed this
    had happened. It now mirrors the primary step and emits a warning when it
    fires.
  • cache-version was unreachable. The restore-key carries no discriminator
    for the dependency set, so a leg can inherit a sibling leg's library — and
    clearing a poisoned cache (e.g. a terra built against the wrong libgdal)
    needed a PR to this repo. Now an input.
  • install-spatial-deps added the ubuntugis-unstable PPA that the
    reusable workflows have explicitly refused for months, because it installs
    libgdal37 against Posit's libgdal34 binaries. The org was shipping two
    contradictory answers to the same question, and SpaDES.tools consumes both.
    Removed; also gains the macOS PROJ_DATA configuration the workflows had and
    this action lacked, plus a version assertion so a partial install fails loudly.
  • install-Require defaulted to GitTag: master, a branch that no longer
    exists on PredictiveEcology/Require. Every caller happens to override it, so
    the default was dead code that would hard-fail if used. Now development.

On timeouts

timeout-minutes is not supported for steps inside a composite action
(actions/runner#1979), and it
is not honoured on the step that calls one either — worth knowing, because
that is the natural thing to reach for. Three alternatives, all used here:

  1. GNU timeout inside the run: block — the apt steps. Gives a
    per-command cap rather than one cap for the whole step, which is strictly
    better than the timeout-minutes: 15 it replaces. Linux/macOS only; Windows
    runners have no GNU timeout.
  2. An action that implements its own timeout — composite actions may uses:
    other actions, so nick-fields/retry's timeout_minutes works. Used for the
    dependency retry.
  3. timeout-minutes on the calling job — a coarse backstop; kills the whole
    job, not the step.

⚠️ Merge order

R-CMD-check.yaml references PredictiveEcology/actions/setup-r-deps@main and
.../stage-gdrive-auth@main. uses: cannot take an expression, so it cannot be
pointed at the PR branch. CI on this PR will fail until the actions exist on
main — that is expected, not a defect in the change. Options:

  • merge on review, then confirm on main; or
  • temporarily point the two uses: lines at
    @refactor/extract-composite-actions, verify green, and revert to @main in
    the same PR before merging.

I have not done the second automatically, since it leaves a footgun if the
revert is forgotten.

Not in scope

test-coverage.yaml, test-downstream.yaml and pkgdown.yaml still carry
their own copies of the setup blocks. They should move to setup-r-deps too,
but doing all four at once makes this unreviewable. Happy to follow up.

🤖 Generated with Claude Code

achubaty and others added 2 commits August 28, 2026 10:52
R-CMD-check.yaml was a 375-line monolith: the dependency setup could not be
reused without also running R CMD check, and the gdrive staging block was
duplicated verbatim in test-coverage.yaml with the two copies already
drifted. Extract both into composite actions and have the workflow call
them; the workflow drops to ~180 lines.

Also align the older composite actions with the choices the workflows
settled on: install-spatial-deps drops the ubuntugis PPA (ABI-incompatible
with Posit's noble binaries, which the workflows have refused for months)
and gains the macOS PROJ_DATA configuration; install-Require's default
GitTag moves off the non-existent master branch.

The retry fallback now mirrors the primary step's dependencies and
extra-packages rather than silently resolving a narrower set.

timeout-minutes is unsupported in composite steps (actions/runner#1979),
so the apt steps use GNU timeout and the dependency retry uses
nick-fields/retry's own timeout_minutes. See NEWS.md.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The migration dropped it, so the matrix produces two jobs both called
`ubuntu-latest (release)`. You cannot tell from the checks list whether
the no-suggests leg passed, branch protection cannot name it as a
required check, and if it stopped being generated nobody would notice.

Appended only when true, so the other job names stay stable for anyone
who has already named them in branch protection.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@achubaty

Copy link
Copy Markdown
Contributor Author

Pushed one more fix onto this branch (0dd2cc0): the nosuggests discriminator is back in the job name.

The migration dropped it, so the matrix currently produces two jobs both called ubuntu-latest (release). That is not cosmetic — you cannot tell from the checks list whether the no-suggests leg passed, branch protection cannot name it as a required check, and if it silently stopped being generated nobody would notice.

name: ${{ matrix.config.os }} (${{ matrix.config.r }}${{ matrix.config.nosuggests && ', nosuggests' || '' }})

Appended only when true, so the other job names stay stable for anyone who has already named them in branch protection.

@achubaty

achubaty commented Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

Tracking the downstream half of this in PredictiveEcology/SpaDES.tools#123 — SpaDES.tools pins these workflows to c2b2459, so it won't pick up the nosuggests job-name fix until this merges and that pin moves.

Noted there, and worth repeating here: v0.5 is not a safe pin. It resolves to 62d556f, which predates e74180e — the commit that fixed the no-suggests leg. Anyone pinning to the newest tag today reintroduces that bug.

@eliotmcintire

Copy link
Copy Markdown
Contributor

@achubaty Thanks — this is the right decomposition. Before it lands, here's where I want the whole ecosystem to end up, since this PR is the first time a reusable workflow here depends on a sibling action and that changes the pinning story.

Pinning: @main everywhere, and give this repo real CI

I don't want tags. A pinned tag always gets old and needs a manual bump in N repos — that's the failure mode we already have. A moving v1 avoids the bumping, but it only buys us a gate between "merged" and "live org-wide," and on a repo this size we'd just move the tag immediately after every merge and get @main with extra ceremony.

The actual problem isn't @main, it's that merging here is unverified: all four workflows are workflow_call:-only, so this repo has no CI. #24 has exactly one check on it and it's GitGuardian. So the plan:

  • Standard pin is @main for both the reusable workflows and the composite actions.
  • This repo gets a self-test — a pull_request-triggered workflow that calls its own reusable workflows against a small fixture package, so a merge to main is actually gated. That's the follow-up that makes @main safe.

Accepted tradeoff: we lose "what exactly ran six months ago." I'm fine with that here.

Merge order for this PR

Rather than the temporary-repoint-and-revert (agreed, that's a footgun), let's split along the seam:

  1. Land setup-r-deps/ + stage-gdrive-auth/ on main alone. Nothing references them yet, so it's inert and can't break a consumer.
  2. Point a branch in SpaDES.core at R-CMD-check.yaml@refactor/extract-composite-actions and get it green.
  3. Merge the R-CMD-check.yaml rewrite.

Same end state, no window where main references something that doesn't exist.

One thing to fix while you're in there: uses: .../setup-r-deps@main inside the workflow defeats a consumer's SHA pin — SpaDES.tools pins the workflow at c2b2459, but the action would resolve from main at runtime. Moot once everyone is on @main, but worth a comment in the file so nobody later assumes a SHA pin is hermetic.

Spatial deps

Agreed on dropping ubuntugis-unstable — two contradictory answers in one org is the bug, and the libgdal37-vs-libgdal34 mismatch is the right tiebreaker. Note NetLogoR pins install-spatial-deps@main, so it picks up both the PPA removal and the new gdal-config --version assertion the moment this merges. That's our canary; let's watch its next run rather than assume.

v0.1/v0.2/v0.4/v0.5 all still carry the PPA, which matters for the list below.

One thing in the extraction itself

stage-gdrive-auth routes its secret through env: with an explicit comment that it is "never pasted into the shell source" — exactly right. But the retry fallback in setup-r-deps interpolates ${{ inputs.dependencies }}, ${{ inputs.extra-packages }} and ${{ inputs.working-directory }} straight into the Rscript -e '...' command. Same PR, both patterns. A single quote in any of those values breaks the command, and it's the injection shape GitHub explicitly warns about. Worth routing through env: for consistency with the sibling action.

Repos that need their pin updated

Already floating, no action: SpaDES, reproducible, SpaDES.core (workflows @main), NetLogoR (install-spatial-deps@main).

Need changing:

repo current
SpaDES.tools workflows @ c2b2459 (all 4) @main
LandR install-spatial-deps@v0.1 @main
map install-spatial-deps@v0.2 @main
SpaDES.shiny install-spatial-deps@v0.2 @main

The three on v0.1/v0.2 are still getting the ubuntugis PPA today.

Also in this repo: the usage examples in all four workflow headers still say @v0.4/@v0.5. Anyone following our own docs onboards straight into both the PPA and the pre-e74180e no-suggests bug. Please flip those to @main in this PR.

Re: your v0.5 warning — correct (it's 62d556f, and e74180e is the very next commit), but I checked and nothing in the org actually pins v0.5, so it's a latent trap rather than a live bug. Once we're all on @main I'd rather delete the stale tags than maintain them.

Service accounts are no longer accepted anywhere in CI. A service account
authenticates cleanly and then cannot upload to user-owned Drive folders, so
upload-backed tests report zero coverage while the job stays green. Staging one
is now a hard failure -- a failed step is the only signal that cannot be
mistaken for "no credential configured". GOOGLEDRIVE_AUTH is no longer exported
to the check/coverage steps; GDRIVE_OAUTH_TOKEN is the contract, which is what
reproducible and SpaDES.project@development already read.

Adds a self-test workflow, the first CI this repo has ever had. The reusable
workflows are workflow_call:-only, so nothing ran on a pull request and every
merge reached the repos pinned at @main unverified. It references the composite
actions by LOCAL path, which resolves to the PR's checkout rather than main, so
a new action can be verified before it exists on main.

Covers setup-r-deps on all three runners (loading terra and reprojecting, since
gdal-config proves only that headers exist, not that the ABI matches),
stage-gdrive-auth across all five credential shapes including the rejected
service account, and install-spatial-deps independently -- it has its own
consumers.

Also routes the setup-r-deps retry fallback's inputs through env: instead of
interpolating them into the Rscript command, matching stage-gdrive-auth, and
points the four usage examples at @main. v0.4 and v0.5 both carry the
ubuntugis PPA and predate e74180e, so the docs were onboarding people into two
known bugs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa
@eliotmcintire

Copy link
Copy Markdown
Contributor

Pushed a847762 onto this branch, folding in the review points above plus two decisions taken since.

What is in it

Service accounts are gone from CI. Not deprecated — rejected, and the rejection fails the step. A service account authenticates cleanly and then cannot upload to user-owned Drive folders, so upload-backed tests report zero coverage while the job stays green; under covr nothing surfaces at all. A warning would have preserved exactly the silent-skip failure mode this PR set out to kill. GOOGLEDRIVE_AUTH is no longer exported to the check/coverage steps; GDRIVE_OAUTH_TOKEN is the contract, which is what reproducible and SpaDES.project@development already read. Verified against yesterday's reproducible run: the org secret is already a user OAuth token, so nothing breaks today.

A self-test workflow — the first CI this repo has ever had. The reusable workflows are workflow_call:-only, so nothing ran on a PR and every merge reached the @main consumers unverified. This PR had exactly one check on it, and it was GitGuardian. It references the composite actions by local path (./setup-r-deps), which resolves to the PR's checkout rather than main — so a new action is verifiable before it exists on main, and the merge-order problem in the description disappears rather than being worked around.

It covers setup-r-deps on all three runners (loading terra and reprojecting — gdal-config --version proves headers exist, only a load proves the ABI matches), stage-gdrive-auth across all five credential shapes including the rejected service account, and install-spatial-deps independently, since it has its own consumers.

The env: fix from the review comment, and the four usage examples repointed to @main.

Merge order

Moot now. The self-test verifies the actions from the PR's own checkout, so this can merge on review.

Next, in order

  1. Merge this, then watch NetLogoR — it pins install-spatial-deps@main, so it is the canary for the ubuntugis removal and the new version assertion.
  2. SpaDES.project@main still calls drive_auth(path = Sys.getenv("GOOGLEDRIVE_AUTH")). Its development branch is already correct. Until that merges, its Drive tests will skip rather than fail — the try(..., silent = TRUE) swallows it. Worth merging development sooner rather than later.
  3. Repin the stragglers off tags and SHAs onto @main: SpaDES.tools (all four workflows at c2b2459), LandR (install-spatial-deps@v0.1), map and SpaDES.shiny (@v0.2). The three on v0.1/v0.2 are still getting the ubuntugis PPA.
  4. Delete the stale tags. v0.4 and v0.5 both carry the PPA and predate e74180e. Nothing in the org pins them, so removing them costs nothing and closes the trap.
  5. Extend the self-test to the reusable workflows. It covers the composite actions only; the workflows check out the caller at its root and run R CMD check there, and this repo has no root DESCRIPTION. Closing that needs a working-directory input threaded through all four — a real change to their public interface, so it is deliberately not in this PR.
  6. test-coverage.yaml, test-downstream.yaml and pkgdown.yaml still carry their own copies of the setup blocks and should move to setup-r-deps, as noted in the description.

Open question, not blocking: reproducible/R/download.R's .gdriveAuthCandidates() still accepts GOOGLEDRIVE_AUTH as a service-account path. That is package runtime rather than CI, so it is a separate call — leaving it alone unless you think end users should lose it too.

eliotmcintire and others added 3 commits August 31, 2026 10:18
…ng CLI

Two bugs the new self-test caught on its first run.

stage-gdrive-auth was broken for every input, not just some. Its `credential`
and `present` descriptions cited `${{ secrets.GOOGLEDRIVE_AUTH }}` as prose
examples, but GitHub evaluates expressions inside `description:` as well, and
`secrets` is not a valid context in a composite action. One such reference
fails the entire manifest to load with "Unrecognized named-value", so the action
never ran at all. Added a note so it is not reintroduced.

The version assertion invoked `proj --version`, but the proj CLI ships in
proj-bin, which libproj-dev does not pull in -- so the check that exists to make
a partial install fail loudly instead failed every complete install on Linux,
and is absent on the macOS runner too. Uses `pkg-config --modversion proj`
instead, whose .pc file ships with the -dev package on both platforms, and adds
pkg-config to the apt list rather than assuming the runner has it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa
The gdrive staging block was ~55 lines duplicated between R-CMD-check.yaml and
test-coverage.yaml, and this PR moved only the first copy to the composite
action. Dropping service-account support then had to be written twice, once in
the action and once in the inline twin -- which is the drift this extraction
exists to prevent, reintroduced in the same change that fixed it.

test-coverage.yaml: 315 -> 250 lines.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa
Its macOS branch is character-for-character the same `brew install pkg-config
gdal` plus PROJ_DATA pair that the setup-r-deps macOS leg already runs. That
brew install pulls ~3GB of bottles (llvm 1.9GB, aws-sdk-cpp 640MB, boost, gcc)
and takes ~200s; macOS runners bill at 10x, so the duplicate cost more than
every Linux job in the run combined while proving nothing new.

Wall clock is unchanged -- the run is still bounded by setup-r-deps (macOS) at
~283s -- so this is a billing fix, not a latency one. Cutting that 283s means
not installing brew gdal at all, which is a behavioural change worth testing
separately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa
@eliotmcintire
eliotmcintire merged commit b2faaa7 into main Aug 31, 2026
10 checks passed
@eliotmcintire
eliotmcintire deleted the refactor/extract-composite-actions branch August 31, 2026 18:20
@eliotmcintire

Copy link
Copy Markdown
Contributor

Correction to my earlier comment: step 1 said "merge this, then watch NetLogoR" as though the run starts by itself. It does not.

None of the five @main consumers — NetLogoR, SpaDES, reproducible, SpaDES.core, SpaDES.project — has workflow_dispatch or schedule. They trigger on push and pull_request only, so merging changes what @main resolves to but starts nothing. To exercise it without waiting for someone to push:

gh run list --repo PredictiveEcology/NetLogoR --workflow R-CMD-check.yaml --limit 1
gh run rerun <id>

Actions at a mutable ref are fetched at job-execution time, so a re-run picks up main as it is now.

Also worth correcting: I listed LandR, map and SpaDES.shiny as "still getting the ubuntugis PPA", which is true but overstates it. The libgdal34/37 ABI mismatch only bites when Posit's prebuilt binaries meet the PPA's libgdal37. LandR and map both set use-public-rspm: false, so they build from source against whatever is present and are self-consistent — repinning them is cleanup. SpaDES.shiny sets use-public-rspm: true, so it is the only one of the three genuinely exposed.

Follow-up in #25: setup-r-deps now delegates its spatial setup to install-spatial-deps rather than carrying a second copy. That could not ride along here, since pre-merge @main was still the copy with the PPA.

eliotmcintire added a commit that referenced this pull request Aug 31, 2026
#25)

The two actions carried two copies of the same apt/brew/PROJ_DATA logic. That
duplication is exactly how the ubuntugis PPA survived in install-spatial-deps
for months after the reusable workflows had rejected it, and it meant the PPA
removal in #24 had to be written twice. One implementation now.

Referenced as PredictiveEcology/actions/install-spatial-deps@main rather than
./install-spatial-deps: inside a composite action a relative `uses:` resolves
against GITHUB_WORKSPACE -- the CALLING repository's checkout -- so the relative
form works only when the caller happens to be this repo and breaks every real
consumer. The self-test cannot catch that, since there the workspace is this
repo and both resolutions coincide.

This could not go in #24: pre-merge, @main was still the copy with the PPA and
without the version assertion, so the self-test would have exercised stale
spatial code and likely failed on the ABI mismatch #24 removes.

setup-r-deps: 250 -> 201 lines. Ports setup-r-deps' stronger
`timeout --kill-after=60s` into the surviving copy so nothing is lost, and drops
the now-obsolete "verified identical by inspection" caveat in the self-test.


Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
eliotmcintire added a commit that referenced this pull request Aug 31, 2026
Blocks the migration of most remaining packages onto these workflows. A survey
of the 17 repos still carrying their own R-CMD-check.yaml found per-package env
this workflow cannot enumerate and had no way to accept: OMP_THREAD_LIMIT
(quickPlot), R_KEEP_PKG_SOURCE (climateData, fpCompare), R_PKG_INSTALL_RETRIES
(SpaDES.shiny), _R_CHECK_THINGS_IN_OTHER_DIRS_ (Require). Without this input
those packages could not migrate without silently dropping settings, and
"silently dropped" is the failure mode this repo keeps having to dig out of.

Newline-separated KEY=VALUE, blank lines and #-comments ignored, applied before
setup-r-deps so it reaches dependency resolution and not just the check. A line
that is not KEY=VALUE fails the step rather than being skipped.

Also refreshes the GOOGLEDRIVE_AUTH secret description, which still advertised
service-account keys as accepted after #24 made them a hard failure.


Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
eliotmcintire added a commit that referenced this pull request Sep 1, 2026
test-coverage does not call setup-r-deps -- it still carries its own inline
pandoc and apt steps -- so the inputs are wired directly onto those steps
rather than passed through. Both default to true; nothing changes for existing
callers.

Smaller payoff than on R-CMD-check: this job is ubuntu-only, so it saves an apt
install rather than the ~3GB macOS Homebrew pull.

Also worth noting the comment above its spatial block is now stale: it warns
that install-spatial-deps uses the ubuntugis PPA, which #24 removed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa
eliotmcintire added a commit that referenced this pull request Sep 1, 2026
* feat(R-CMD-check): expose system-deps and pandoc to callers

setup-r-deps has had both switches since it was extracted, but the reusable
workflow neither exposed nor passed them, so every caller installed the
geospatial stack and pandoc whether or not it could use them.

Six packages in the org declare no spatial dependency at all -- fpCompare,
Require, pedev, peutils, SpaDES.install, SpaDES.addins -- and each was pulling
~3GB of Homebrew (llvm, gcc, boost, aws-sdk-cpp, ~200s) on its macOS leg at the
10x macOS billing rate, plus an apt install on every Linux leg, for libraries
they never load.

Both default to true, so no existing caller changes behaviour.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa

* feat(test-coverage): honour system-deps and pandoc too

test-coverage does not call setup-r-deps -- it still carries its own inline
pandoc and apt steps -- so the inputs are wired directly onto those steps
rather than passed through. Both default to true; nothing changes for existing
callers.

Smaller payoff than on R-CMD-check: this job is ubuntu-only, so it saves an apt
install rather than the ~3GB macOS Homebrew pull.

Also worth noting the comment above its spatial block is now stale: it warns
that install-spatial-deps uses the ubuntugis PPA, which #24 removed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012DVjmY3im9Xak7tXLSMGCa

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants