Skip to content

feat(pr-review): make the review pipeline inspectable - #14

Merged
Troublor merged 1 commit into
mainfrom
william/refactor/pr-review
Aug 24, 2026
Merged

feat(pr-review): make the review pipeline inspectable#14
Troublor merged 1 commit into
mainfrom
william/refactor/pr-review

Conversation

@Troublor

Copy link
Copy Markdown
Collaborator

Summary

The PR review CI was effectively a black box. prepare, compose, compile, and publish printed nothing but failure annotations, the analysis session's output was hidden behind a debug_logs input defaulting to false, and everything a run produced — review-input.json, both diffs, the raw model output, the compiled payload — lived in .pr-review/ on a runner that then disappeared. When a review reached a surprising verdict, or a finding the model reported never appeared on the PR, there was no way to tell why.

This adds three inspection surfaces, all derived from the same trace lines so they cannot disagree.

Job log — collapsible groups, in step order:

  • Review routing (prepare) — mode and why, prior-state source and whether its version still matches, previous and current head, compare status and file count, both diff sizes, how much of the PR conversation was included versus truncated, the paths in scope, and every open prior finding and question handed to the model.
  • Analysis settings (compose) — model, tier, depth, pre-mortem, turn budgets, allowed tools.
  • The analysis session's full output, now always printed.
  • Model output (raw, before compilation) and Compilation decisions (compile) — one line per model result the compiler accepted, suppressed as a duplicate of an open finding, dropped at the per-severity cap of five, or rerouted to the review body because its line is not commentable, plus each prior finding and question disposition. These drops were previously silent and are the usual reason a finding exists in the model's output but never reaches the PR.
  • Publication (publish) — review ID, inline comments requested versus posted, inline fallback, threads resolved, sticky comment ID.

Step summary — routing decision and reason, head range, the model and turn budget the analysis actually ran with, tier/high-risk/pre-mortem flags, and this round's counts (findings split inline versus review body, new questions, prior findings resolved), then folds in the pipeline trace and the raw pre-compilation model output. A failed run gets the same trace, showing how far the round got.

Run artifact pr-review-state-<pr>-<run>-<attempt> (14 days by default) carries the bytes themselves: routing input, both diffs, analysis-transcript.json (the session's turn-by-turn record, plus a retry twin), raw and parsed structured output, compiled payload, publish result, and trace.log. Upload needs no new permissions — it uses the Actions runtime token.

Also: when the structured output fails to parse, the unparsable bytes are now kept as model-output.raw. That is exactly the case where they matter, and compile previously died holding the only copy.

Interface changes

  • debug_logs is deprecated and ignored; show_full_output is now unconditionally true. Consumers still passing it are unaffected.
  • New optional inputs state_artifact (default true) and state_artifact_retention_days (default 14).
  • actions/upload-artifact pinned to 043fb46 (v7.0.1).

Test plan

  • python3 -m unittest discover -s .github/actions/claude-pr-review -p 'test_*.py' — 101 tests pass (was 85). New coverage: trace emission and persistence, list truncation, the routing explanation including a truncated conversation, compilation traces for accepted/rerouted/capped/deduped findings and truncated questions, summary detail blocks and their size cap, the analysis-settings rows, and the enriched success report.
  • actionlint on the workflow, bash -n on every composite run block, and yq parse of action.yml.
  • End-to-end: after merge, open any PR in a consumer repo and check the run's step summary, the new log groups, and the pr-review-state-* artifact.

Log the routing decision, analysis settings, every compilation choice, and
the publication result; always print the analysis session output; and upload
the .pr-review state directory and session transcript as a run artifact.
@Troublor
Troublor merged commit 6c3bff5 into main Aug 24, 2026
1 check passed
@Troublor
Troublor deleted the william/refactor/pr-review branch August 24, 2026 01:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant