Skip to content

fix(ci): stop AI PR Review from failing a successful review - #20

Merged
shawnmuggle merged 1 commit into
mainfrom
fix/ai-pr-review-red-on-success
Sep 1, 2026
Merged

fix(ci): stop AI PR Review from failing a successful review#20
shawnmuggle merged 1 commit into
mainfrom
fix/ai-pr-review-red-on-success

Conversation

@shawnmuggle

Copy link
Copy Markdown
Member

AI PR Review has failed 26 times across 4 repos in the last 72h while the review itself succeeded every time. Two independent causes, both in the wrap-up after the review is already posted:

  • v1 (ai_pr_review.py): the auto-approve POST returns 401 Unauthorized (AI_REVIEW_APPROVER_TOKEN expired or under-scoped) and raises, after the review comment and pass label have already landed. Now caught: the run degrades to "reviewed, not approved" and prints the HTTP code and likely cause in the step summary.
  • v2 (ai-pr-review-v2.yml): --max-turns 15 is below what a real review costs — gh pr diff + per-file reads + one inline comment per finding. Observed Claude reported a successful result after 18 turns and error_max_turns (no structured_output) on a 3-file PR. Raised to 40; spend is bounded by the daily USD cap, not by this number.

Applied identically across all 16 repos carrying these workflows (v1 script sha was d7a86c89 everywhere, v2 workflow sha 4f88ddb9 everywhere).

Note: the 401 itself is a separate follow-up — the approver PAT still needs rotating for auto-approve to resume.

The review comment and pass label are posted before the approval call, so
a dead AI_REVIEW_APPROVER_TOKEN was turning a completed review into a red
check. Degrade to "reviewed, not approved" and surface the HTTP code in
the step summary instead of raising.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-01T12:26:12.738457Z 454238e PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@shawnmuggle
shawnmuggle merged commit 552fd79 into main Sep 1, 2026
1 check passed
@shawnmuggle
shawnmuggle deleted the fix/ai-pr-review-red-on-success branch September 1, 2026 12:25

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 454238eba5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/ai_pr_review.py
f"Review posted, but auto-approve failed "
f"(HTTP {e.code}). {hint}"
)
except urllib.error.URLError as e:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Treat response-body timeouts as approval failures

When GitHub sends response headers but stalls while gh_request() is reading the body, resp.read() can raise TimeoutError directly rather than urllib.error.URLError. That exception still escapes this new best-effort boundary and makes an otherwise completed review fail—the exact behavior this change is intended to prevent. Catch TimeoutError here as a network failure as well.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant