Skip to content

ci: skip stacked PRs unless they carry ci/force-ci - #449

Merged
raphaelvigee merged 3 commits into
masterfrom
raphaelvigee/ci-stack-force-ci
Sep 3, 2026
Merged

raphaelvigee merged 3 commits into
masterfrom
raphaelvigee/ci-stack-force-ci

Conversation

@raphaelvigee

@raphaelvigee raphaelvigee commented Aug 31, 2026 •

Copy link
Copy Markdown
Member

Three repo-config changes.

1. Drop the Orca default worktree tabs

orca.yaml's defaultTabs (a claude tab wrapped in devenv shell, opened in every new worktree) is gone. scripts.setup is untouched, so a new worktree still gets gen — which it cannot build without, since gen/proto is a declared workspace member whose Cargo.toml isn't committed — and a codegraph index.

What goes with it is the wrapping. Anything opening a shell or agent in a worktree now has to enter the devenv shell itself, and the failure mode when it doesn't is the quiet one: missing cargo/buf/gen/lint fails immediately and visibly, but an ambient rustc with no RUSTC_WRAPPER builds perfectly well and silently bypasses kache. The header comment keeps the command and the reason.

2. Stacked PRs no longer run CI

A PR stacked on another PR's branch is not the change about to land. It can't merge until everything under it does, and master is squash-only so the layer above always gets rebased once its base lands — that force-push is the run anyone acts on. Building every layer on every push multiplied a deep stack's cost by its depth for verdicts about a tree nobody would merge.

A new gate job decides once:

situation builds?
push to master yes
PR whose base is the default branch yes
PR whose base is another branch no
…unless it carries ci/force-ci yes

gen, govet and abi hang off it; everything else is transitively behind those. A skipped PR reports skipped checks and costs one ~15s runner.

The stack still merges, with no ruleset change

Required status checks live in the master ruleset, whose condition is ~DEFAULT_BRANCH — it applies to exactly the PRs whose base is master, which is exactly the set the gate builds. GitHub asks nothing of a PR that doesn't target a protected branch.

That argument has one seam, and edited in the trigger list is what closes it. A job skipped by an if: concludes skipped, which branch protection reads as a pass. GitHub retargets a child PR at master by itself when the base merges, and that retarget fires no synchronize — so without edited the child would sit at master carrying skipped-reads-as-green checks over content nothing ever built. The retarget is an edited event with changes.base, so it now starts a run whose base is master. gh stack sync (required anyway) is the second path; neither is trusted alone.

Cost of the extra trigger types, taken deliberately

labeled and edited fire on things that aren't code changes, and the gate can't suppress those without concluding "skip" on a master-targeting PR — which would replace its green required checks with skipped ones. So a label or a body edit on a master-targeting PR re-runs CI, and with cancel-in-progress doing either mid-run restarts it. gh stack submit pushes and rewrites bodies seconds apart, so those collapse into one run via the concurrency group.

Diagnosability

The summary job reports the verdict on the run page. The gate itself writes its reason to stdout and a ::notice annotation, and deliberately not to the step summary — $GITHUB_STEP_SUMMARY renders on the run page and nowhere else, not in gh run view --log and not in gh pr checks --json, so an agent seeing eleven skipping rows and exit 0 couldn't reach it. The annotation is in the check-run API and survives into gh. The text also names the ci/perf-test trap (labelling a stacked PR for perf gets nothing — perfbench needs the gated build). summary branches on needs.gate.result before touching its outputs, so a cancelled gate isn't reported as "this PR is stacked".

Tests

tests/ci_gate.rs freezes the structure, not today's job list: every job reachable from gate; a job that needs gate must also read its verdict (otherwise it looks gated and isn't); labeled/edited stay in the trigger types; the gate emits an annotation; summary consults gate.result; and no branches: filter returns to pull_request: — that skips at the workflow level, which publishes no checks at all, an empty list that reads like a pass (#240). Each guard was mutation-tested.

Also fixes a comment this change falsified: the perfbench block said adding a label doesn't retrigger the job.

3. perf-test → ci/perf-test

Both CI labels now share one namespace — a lone pr/ prefix on the other was a typo waiting to happen. Renamed in place on the repo, so ci/perf-test keeps its id and stays applied to #435 and #437. The functional reference is perfbench's if:; the rest are comments here and in perf.yml.

Not touched: the perf-test skill in .claude/skills/ and its references in .claude/agents/ and .cargo/config.toml — that's samply profiling, unrelated beyond sharing a name.

One transition window: a pull_request run resolves its workflow from head-merged-into-base, so an open PR carrying the label gets the renamed if: as soon as this lands on master, no rebase needed. Before it lands, those two PRs hold a label their own base's workflow doesn't match, so perfbench won't fire for them in that window.

Still open

Full skip vs. reduced matrix. product-vision offered running lint + test on linux/amd64 for stacked PRs — ~⅓ the spend, and it keeps real green checks plus pre-merge platform signal instead of a wall of skips. Full skip means a break that only shows on darwin/arm64 surfaces only once the layers below land. Recorded in CLAUDE.md next to the escape hatch rather than decided quietly.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NQ8kRpuBjgxsSf15RjLDZz

`defaultTabs` opened a `claude` tab, wrapped in `devenv shell`, in every
new Orca worktree. It is gone; `scripts.setup` is untouched, so a new
worktree still gets `gen` (which it cannot build without) and a
`codegraph` index.

What goes with it is the wrapping. Anything that opens a shell or an
agent in a worktree now has to enter the devenv shell itself, and the
failure mode when it does not is the quiet one rather than the loud one:
missing `cargo`/`buf`/`gen`/`lint` fails immediately and visibly, but an
ambient `rustc` with no `RUSTC_WRAPPER` builds perfectly well and simply
bypasses kache. The header comment keeps the command and the reason so
neither has to be rediscovered.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NQ8kRpuBjgxsSf15RjLDZz
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/ci-stack-force-ci branch 2 times, most recently from 8adec41 to c1676df Compare September 2, 2026 09:56
raphaelvigee and others added 2 commits September 2, 2026 12:14
A PR stacked on another PR's branch is not the change that is about to
land. Everything under it has to merge first, and `master` is
squash-only, so the layer above always gets rebased once its base lands
— that force-push is the run whose result anyone acts on. Building every
layer on every push multiplied a deep stack's CI cost by its depth to
produce verdicts about a tree nobody would ever merge.

A new `gate` job decides once: a push, or a PR targeting the default
branch, builds; anything else builds only with the `ci/force-ci` label.
`gen`, `govet` and `abi` hang off it and every other job is transitively
behind those, so a skipped PR costs one ~15s runner.

The stack stays mergeable with no ruleset change. Required status checks
live in the `master` ruleset, whose condition is `~DEFAULT_BRANCH`, so
they apply to exactly the PRs the gate builds; GitHub asks nothing of a
PR that does not target a protected branch.

That argument has one seam, and `edited` in the trigger list is what
closes it. A job skipped by an `if:` concludes `skipped`, which branch
protection reads as a *pass* — so a child PR that became
master-targeting without a new run would be mergeable carrying skipped
checks over content nothing ever built. GitHub's automatic retarget is
an `edited` event with `changes.base`, which triggers a run whose base
is now master. `gh stack sync` (required anyway) is the second path.
Neither is trusted alone.

Cost of the two extra trigger types, taken deliberately rather than
glossed: `labeled` and `edited` fire on things that are not code
changes, and the gate cannot suppress those without concluding "skip" on
a master-targeting PR, which would replace its green required checks
with skipped ones. So a label or a body edit on a master-targeting PR
re-runs CI, and with `cancel-in-progress` doing either mid-run restarts
it. `gh stack submit` pushes and rewrites bodies seconds apart, so those
collapse into one run.

The gate writes its reason to stdout and to a `::notice` annotation, and
deliberately not to the run-page step summary — that belongs to the
`summary` job, which already reports the verdict there. The two surfaces
it does write are the ones a non-browser reader can reach:
`$GITHUB_STEP_SUMMARY` renders on the run page and nowhere else, so an
agent reading `gh pr checks` — eleven `skipping` rows and exit 0 — could
not reach it, while the annotation is in the check-run API. `summary`
branches on `needs.gate.result` before its outputs, so a cancelled gate
is not reported as "this PR is stacked".

`tests/ci_gate.rs` freezes the structure rather than today's job list:
every job must be reachable from `gate`, a job that needs `gate` must
also read its verdict, `labeled`/`edited` must stay in the trigger
types, the gate must emit an annotation, `summary` must consult
`gate.result`, and no `branches:` filter may return to `pull_request:`
(that one skips at the workflow level, which publishes no checks at all
— an empty list that reads like a pass, #240). Each guard was
mutation-tested.

Reviewed by `product-vision`; two of its recommendations are left to the
user rather than taken here. The label name `ci/force-ci` is the user's,
and disagrees with the incumbent bare `perf-test`. And full-skip was
chosen over a reduced stacked-PR matrix (lint + test on linux/amd64,
~1/3 the spend, real green checks and pre-merge platform signal): a
break that only shows on darwin/arm64 now surfaces only once the layers
below land. That per-target trade is recorded in CLAUDE.md next to the
escape hatch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NQ8kRpuBjgxsSf15RjLDZz
Renamed in place on the repo (`gh label edit`), so it keeps its id and
stays applied to the PRs already carrying it — #435 and #437 both still
have it. With `ci/force-ci` from the commit below, both CI labels now
share one namespace; a lone `pr/` prefix on one of them was a typo
waiting to happen.

The functional reference is `perfbench`'s `if:`; the rest are comments
here and in `perf.yml`. Not touched: the `perf-test` *skill* in
`.claude/skills/` and the references to it in `.claude/agents/` and
`.cargo/config.toml`. That is samply profiling, unrelated to the label
beyond sharing a name.

One transition window worth knowing about. A `pull_request` run resolves
its workflow from the merge of head into base, so an open PR carrying
the label gets the renamed `if:` as soon as this lands on master, with
no rebase needed. Before it lands, those PRs hold a label the workflow
on their own base does not match — perfbench will not fire for them in
that window. It is an opt-in check on two PRs, so the cost is a re-label
or a rebase, not a lost signal.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NQ8kRpuBjgxsSf15RjLDZz
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant