Skip to content

refactor(ci): route every reusable workflow through setup-r-deps; drop the retry fallback - #28

Closed
achubaty wants to merge 7 commits into
mainfrom
ci/route-workflows-through-setup-r-deps
Closed

refactor(ci): route every reusable workflow through setup-r-deps; drop the retry fallback#28
achubaty wants to merge 7 commits into
mainfrom
ci/route-workflows-through-setup-r-deps

Conversation

@achubaty

@achubaty achubaty commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Waves 1a–1e of the CI remediation plan. These all touch the same handful of
files, so they are one unit of work, split into six commits.

1a — pkgdown, test-coverage and test-downstream now go through setup-r-deps

Only R-CMD-check.yaml called the composite action. The other three each
carried their own inline apt block, their own setup-pandoc (with three
different retry counts
: 3 in setup-r-deps, 1 in test-downstream, 0 in
pkgdown/test-coverage), and their own hardcoded
extra-repositories: 'https://PredictiveEcology.r-universe.dev/'.
test-downstream.yaml also carried a stale copy of the dependency retry
fallback that dropped inputs.extra-packages — the bug fixed in setup-r-deps
in #24 and never backported.

The side effect that matters most: all three now get the RcppParallel/oneTBB
rebuild probe
that only R-CMD-check had. A package built against the old TBB
ABI installs without complaint and only fails at dlopen, so on the coverage
job that was covr dying with no coverage and no useful message.

Genuine per-workflow behaviour is preserved:

workflow preserved
test-coverage run-in-tmux, and the entire covr/codecov step verbatim
pkgdown local::. + needs: website, the deploy step, contents: write
test-downstream the Remotes: unpin and the RemoteSha assertion

The RemoteSha assertion step is byte-identical (verified by diffing the
parsed YAML against main). The unpin script is byte-identical apart from the
environment variable it reads (PKGPKG_UNDER_TEST), and I ran the
extracted script against a synthetic downstream DESCRIPTION with a one-line
Remotes: field to confirm the YAML round-trip preserved it.

Two ordering details in test-downstream:

  • Identify package under test stays deliberately shell-only and moves ahead
    of the dependency setup. It now also exports the name via $GITHUB_ENV,
    because a composite action's steps cannot see the caller's step-level env:.
  • The unpin runs as setup-r-deps' new pre-install hook, i.e. still after
    setup-r and still before pak resolves anything. That window is the only
    place it works, and it is now inside the action rather than between two of
    the job's steps.

test-coverage's tmux install moves after setup-r-deps (which is what runs
apt-get update) and now runs its own apt-get update, so system-deps: false
cannot break it.

1b — system-deps and pandoc as workflow_call inputs

setup-r-deps has gated its "Install geospatial system dependencies" step on
system-deps == 'true' since it was extracted, but no reusable workflow ever
forwarded the input, so no caller could turn it off. Both are now inputs on
R-CMD-check, test-coverage and test-downstream, defaulting to true.
system-deps is also on pkgdown; pandoc deliberately is not, because pkgdown
cannot render a site without it.

Not auto-derived from the DESCRIPTION, as the plan requires: reproducible
and SpaDES.project reach terra/sf/raster/geodata through Suggests only, so
any rule based on hard dependencies would decide "pure R" for exactly the two
packages that need GDAL most.

1c — cache-version

Now an input on pkgdown and test-coverage, matching R-CMD-check. Neither could
evict a poisoned dependency cache without a PR to this repo, and a stale cache
was the root cause of both historical libgdal ABI incidents.

Slight scope extension, flagged for review: I added it to
test-downstream.yaml too, plus extra-repositories on all three. The
cache-version rationale applies identically to test-downstream (the plan says
"the only two", which I think undercounts by one), and exposing
extra-repositories with the r-universe default it already hardcoded is what
made the consolidation behaviour-preserving rather than a change. Happy to drop
either if you disagree.

1d — false comments

  • The "r-universe terra is libgdal37" claim is gone from pkgdown and
    test-coverage (both comment blocks were deleted wholesale by 1a). It was never
    true: r-universe's base image has never contained ubuntugis, pinned
    FROM ubuntu:noble in May 2026 (libgdal.so.34, matching PPM) and is .38
    today.
  • The ABI comment in install-spatial-deps no longer blames quickPlot.
    quickPlot is NeedsCompilation: no (confirmed: no src/, and
    packageDescription("quickPlot", fields = "NeedsCompilation") is no) and
    links nothing — it merely happens to be the package that loads sf/terra
    first.
  • libgdal34libgdal34t64, which is noble's package name;
    libgdal.so.34 is the soname it provides, so the error string is unchanged.
  • Most importantly, install-spatial-deps now says why it still exists, in
    the action and in its README. It exists for the SpaDES module
    repositories: no DESCRIPTION (dependencies live in
    defineModule(reqdPkgs = ...)), installation via Require::Require() rather
    than pak, and Require's .onLoad deliberately sets PKG_SYSREQS=false /
    PKG_SYSREQS_SUDO=false because CRAN treats a package shelling out to sudo
    as machine hijacking. Verified empirically against Require 2.0.0.9036:
    loading the package sets both variables to false. On that path nothing else
    installs GDAL. Without the note, the action reads as dead code now that
    setup-r-deps covers the package repos.

1e — the retry fallback, and permissions:

The fallback was never a retry. The primary path is pak::lockfile_install()
(upgrade = FALSE); the fallback called pak::local_install_dev_deps(), whose
default is upgrade = TRUE — bypassing both the lockfile and the cache to
resolve a more aggressive set than the one that had just failed. Paired with
continue-on-error: true on the primary step, a genuinely broken dependency
bought ~90 minutes of futile reinstallation instead of an immediate red.
continue-on-error is deleted with it, so a failed install is now red at once.

It is also unnecessary. From the installed pak's own NEWS:

pak now retries failed HTTP requests by default, using an exponential backoff
(and honoring the Retry-After header).

That entry is under the # pak 0.11.0 heading, not 0.11.1 as the plan
states — 0.11.1 is simply the release that carries it (CRAN, 2026-07-22), and
setup-r-dependencies defaults to pak-version: stable. The conclusion is
unaffected; noting the version discrepancy for the record.

permissions: blocks: none of the four reusable workflows declared any, so a
caller inheriting an org default of read/write handed a writable GITHUB_TOKEN
to every step — including the third-party packages post-install fetches from
GitHub. Three get contents: read; pkgdown's job gets contents: write, which
its gh-pages deploy genuinely needs.

Not read-all, despite the plan (and r-lib) suggesting it — this would have
broken SpaDES.tools.
A called workflow whose permissions: request exceeds
what the caller's job granted fails validation, before the run starts
(The workflow is requesting 'actions: read', but is only allowed 'actions: none'), rather than being silently clamped. I audited every caller in
the org: SpaDES.tools' thin callers set permissions: {} at file level and
re-grant exactly contents: read on the job, so read-all — which requests
read on every scope — would have taken down its R-CMD-check, test-coverage
and test-downstream on their next run. read-all is correct for a standalone
workflow like r-lib's templates; it is not correct here. contents: read is
what the checkout needs, and is a subset of every current caller's grant.
See commit ffaa23b; the intermediate read-all version is left in history
deliberately, since the reasoning is the useful part.

self-test

self-test.yaml cannot reach the reusable workflows (its header explains why:
no DESCRIPTION at this repo's root), but it does cover the composite actions,
and both new setup-r-deps inputs are now exercised. An unknown input to a
composite action is a warning, not an error, so a rename would otherwise
surface as a pkgdown build quietly missing its site tooling — not as anything
red.

  • pre-install asserts positively that it runs before dependency resolution:
    it fails if terra, sessioninfo or abind are already installed at that point,
    and drops a marker a later step checks.
  • needs is covered by a new Config/Needs/self-test: abind in the fixture.
    abind is in nothing else's dependency closure here, so its presence can only
    have arrived through that input.

Validation

  • actionlint 1.7.12 clean on the whole repo (clean on main too, so this is a
    like-for-like comparison).
  • Every YAML file parses (yaml.safe_load).
  • Every embedded R script extracted from every workflow and action and
    syntax-checked with parse().
  • The extracted unpin script executed against a synthetic downstream
    DESCRIPTION; read.dcf() round-trips the rewritten file.

Risks a human should look at

  1. The apt package set shifts slightly for the three consolidated
    workflows. install-spatial-deps adds pkg-config, uses
    --no-install-recommends, and drops the apt-get install -f -y /
    --fix-missing belt-and-braces the inline blocks had. Intended, but it is a
    real behaviour change on three workflows at once.

  2. pre-install interpolates a ~35-line R script through a composite action
    input.
    It works and is tested, but it is not pretty; the alternative was
    rewriting the unpin in shell, which I would not do to that particular code.

  3. Draft on purpose: the reusable workflows reference
    PredictiveEcology/actions/setup-r-deps@main, so needs and pre-install
    do not exist for them until this merges. The self-test uses the local path
    and does cover the branch's copy, but the workflows themselves are only
    proven by a consumer's first run after merge. Worth watching one
    test-downstream and one pkgdown run.

  4. pkgdown now fails earlier when a caller under-grants. Declaring
    contents: write means a caller whose job does not grant it fails
    validation up front, instead of running the whole build and failing at the
    deploy step. That is better behaviour, but it is a behaviour change: it
    assumes the four callers that declare no permissions: at all
    (SpaDES.core, reproducible, SpaDES, SpaDES.project) sit on a repository
    default of read/write. They must today, or their deploys would already be
    failing — worth a sanity check before this leaves draft.

  5. Caller audit, for the record. Only SpaDES.tools declares permissions
    on its caller jobs (contents: read ×3, contents: write for pkgdown);
    SpaDES.core, reproducible, SpaDES, SpaDES.project, NetLogoR and map declare
    none and inherit the repository default. No caller passes an input this PR
    removes, because it removes none.

🤖 Generated with Claude Code

achubaty and others added 7 commits August 31, 2026 17:47
…needs

The "retry fallback" was never a retry. The primary path is
pak::lockfile_install() (upgrade = FALSE); the fallback called
pak::local_install_dev_deps(), whose default is upgrade = TRUE -- so it
bypassed both the lockfile and the cache to resolve a MORE aggressive set than
the one that had just failed. Paired with continue-on-error: true on the
primary step, a genuinely broken dependency bought ~90 minutes of futile
reinstallation instead of an immediate red.

It is also unnecessary now. pak retries failed HTTP requests by default (pak
0.11.0 NEWS: exponential backoff, honouring Retry-After, covering package
downloads and not just metadata) and pak-version defaults to `stable`, i.e. the
current CRAN release (0.11.1, 2026-07-22). The transient network failure the
fallback was written for is handled one layer down now. continue-on-error goes
with it, so a failed dependency install is red immediately.

Two new inputs, both needed to route the remaining reusable workflows through
this action:

  * `needs` forwards setup-r-dependencies' own `needs`, i.e.
    Config/Needs/<field>. pkgdown.yaml uses `website` and would otherwise lose
    its site tooling.

  * `pre-install` is an Rscript run after setup-r and BEFORE the dependency
    install. test-downstream.yaml has to rewrite the downstream package's
    Remotes: field in exactly that window -- any earlier and there is no R to
    rewrite the DESCRIPTION with, any later and pak has already resolved the
    pinned upstream from GitHub -- so without this hook that workflow could not
    use this action at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…r-deps

Only R-CMD-check.yaml called setup-r-deps. pkgdown, test-coverage and
test-downstream each carried their own inline apt block, their own setup-pandoc
(with three different retry counts: 3, 1, 0), and their own hardcoded
extra-repositories. test-downstream also carried a stale copy of the dependency
retry fallback that dropped extra-packages -- a bug fixed in setup-r-deps in
#24 and never backported, and since deleted outright.

All three now pick up the RcppParallel/oneTBB rebuild probe that only
R-CMD-check had. That matters: a package built against the old TBB ABI installs
without complaint and fails at dlopen, which on the coverage job means covr
dying with no coverage and no useful message.

Genuine per-workflow behaviour is preserved:

  * test-coverage keeps run-in-tmux and the whole covr/codecov step verbatim.
    Its tmux install moves after setup-r-deps (which is what runs apt-get
    update) and now runs its own update, so system-deps: false cannot break it.
  * pkgdown keeps local::. + needs: website, its deploy step, and gains
    contents: write scoped to the job. `pandoc` is deliberately NOT exposed
    there: pkgdown cannot render without it.
  * test-downstream keeps the Remotes: unpin -- byte-identical apart from the
    env var it reads -- now passed as setup-r-deps' pre-install so it still
    runs after setup-r and before pak resolves anything. The RemoteSha
    assertion is untouched. "Identify package under test" stays shell-only and
    moves ahead of the dependency setup, exporting the name via GITHUB_ENV
    because a composite action's steps cannot see the caller's step-level env:.

New inputs on all three, matching R-CMD-check: cache-version (the two libgdal
ABI incidents were both a stale cache, and none of these workflows could evict
one without a PR to this repo), extra-repositories (same default as before, so
callers are unaffected), system-deps and pandoc.

permissions: read-all on test-coverage and test-downstream, matching
r-lib/actions' own templates; pkgdown's job takes contents: write for the
gh-pages deploy. None of the three declared permissions before, so a caller
whose repository default is read/write handed them -- and every third-party
package they install -- a writable GITHUB_TOKEN.

Deletes two comments that were never true: r-universe's terra is not built
against libgdal37 (its base image has never carried ubuntugis, pinned
FROM ubuntu:noble in May 2026, and is .38 today).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ermissions

setup-r-deps has gated its "Install geospatial system dependencies" step on
system-deps == 'true' since it was extracted, but the reusable workflow never
forwarded the input, so no caller could actually turn it off. Same for pandoc.
Both default to true, so nothing changes for existing callers.

Deliberately NOT auto-derived from the DESCRIPTION. reproducible and
SpaDES.project reach terra/sf/raster/geodata through Suggests only, so a rule
based on hard dependencies would decide "pure R" for exactly the two packages
that need GDAL most.

permissions: read-all, matching r-lib/actions' own check templates. The
workflow declared none, so a caller inheriting an org default of read/write
handed a writable GITHUB_TOKEN to every step -- including the third-party
packages post-install installs from GitHub.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
setup-r-deps installs the same libraries, by delegating to this action, so from
inside the package-CI world this now reads as dead code. It is not, and the
reason was written down nowhere.

The SpaDES *module* repositories cannot go through the package path. A module
repo has no DESCRIPTION -- dependencies are declared in
defineModule(reqdPkgs = ...) -- so pak and setup-r-dependencies have nothing to
resolve against, and modules install with Require::Require() instead. Nothing
on that path installs system libraries: Require's .onLoad sets
PKG_SYSREQS=false and PKG_SYSREQS_SUDO=false (verified against Require
2.0.0.9036), deliberately, because CRAN treats a package that shells out to
sudo on the user's machine as machine hijacking. So on a module repo's CI,
GDAL/GEOS/PROJ arrive from this action or not at all.

Also corrects the ABI comment. It blamed quickPlot, which is
NeedsCompilation: no and links nothing -- it merely happens to be the package
that loads sf/terra first. And noble's package is libgdal34t64, not libgdal34;
libgdal.so.34 is the soname it provides.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An unknown input to a composite action is a WARNING, not an error. A rename or
typo in either of the new inputs would therefore surface as a pkgdown build
quietly missing its site tooling, or as test-downstream silently checking the
wrong code -- the exact failure mode this repository keeps having to dig out
of, and neither would be red.

pre-install asserts positively that it runs BEFORE dependency resolution: it
fails if terra, sessioninfo or abind are already installed when it runs, and
drops a marker a later step checks. needs is covered by a new
Config/Needs/self-test: abind in the fixture -- abind is in nothing else's
dependency closure here, so its presence can only have come through that input.

The reusable workflows themselves remain outside this workflow's reach, as its
header already documents.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Also rewrites the two development-section bullets the change contradicts: the
setup-r-deps summary no longer advertises a retry fallback, and the bullet that
described fixing that fallback now records its removal.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
read-all is what r-lib/actions' own templates use, but it is wrong for a
REUSABLE workflow. A called workflow whose permissions request exceeds what the
caller's job granted fails validation before the run starts -- "The workflow is
requesting 'actions: read', but is only allowed 'actions: none'." -- rather than
being silently clamped.

SpaDES.tools' thin callers set `permissions: {}` at the file level and re-grant
exactly `contents: read` on the job. read-all requests read on every scope, so
merging it would have taken down that repo's R-CMD-check, test-coverage and
test-downstream on their next run.

contents: read is what the checkout needs, is strictly less than read-all, and
is a subset of every current caller's grant: SpaDES.tools grants exactly it,
and the callers that declare nothing inherit a read/write default that
contains it. Artifact upload authenticates with ACTIONS_RUNTIME_TOKEN and
codecov with CODECOV_TOKEN, so neither needs more.

pkgdown keeps contents: write -- SpaDES.tools' pkgdown caller grants exactly
that, and the others inherit read/write.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@achubaty

Copy link
Copy Markdown
Contributor Author

Withdrawing this — PredictiveEcology/actions is under active rework by its owner, so an external PR here would be superseded rather than reviewed. Closing to keep the repo's PR queue clean.

The branch ci/route-workflows-through-setup-r-deps is left in place, so nothing is lost if any of it is wanted later. The substantive findings behind it, for the record:

  • test-coverage, pkgdown and test-downstream each carry their own copy of the dependency setup that setup-r-deps already provides — including three setup-pandoc invocations with three different retry counts (3/1/0), three hardcoded extra-repositories, and a copy of the retry fallback in test-downstream that still drops inputs.extra-packages (a bug already fixed in setup-r-deps). They also never run the RcppParallel/oneTBB rebuild probe.
  • setup-r-deps declares system-deps and pandoc, but no reusable workflow forwards them, so no caller can opt out of the geospatial install.
  • pkgdown.yaml and test-coverage.yaml expose no cache-version, and a stale package cache was the actual root cause of both historical libgdal ABI incidents.
  • Worth knowing before anyone adds permissions: to the reusable workflows: a called workflow requesting more than the caller's job granted fails validation before the run starts, it is not silently clamped. permissions: read-all would therefore have broken SpaDES.tools, whose thin callers set permissions: {} and re-grant exactly contents: read. contents: read is the safe value for a reusable workflow; read-all is right only for standalone templates.

🤖 Generated with Claude Code

@achubaty achubaty closed this Aug 31, 2026
@achubaty
achubaty deleted the ci/route-workflows-through-setup-r-deps branch September 1, 2026 00:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant