FF #2900: bot-review-gate ack fix — real-body fixtures, replay guard, fenced red, trailer - #2901
Merged
Merged
Conversation
…, blacklist acks and failure notices only - add is_coderabbit_walkthrough() that classifies an auto-summary issue comment as a real review iff it carries a Run ID and at least one signal (quota-decrement line, no-actionable phrase, or Files-processed list) - remove auto-summary marker from is_coderabbit_scaffolding blacklist - add is_coderabbit_failure_notice() and blacklist only ack + failure notice - keep rate-limit stub detection unchanged - run scaffolding check after APPROVED/CHANGES_REQUESTED in is_real_item() - fold the two stub branches into one accurate message - red-proof with real-body fixtures: walkthrough control, ack-only, and failure-notice-only tests; replay guard from merged-PR shapes
Replace synthesized walkthrough fixtures with genuine CodeRabbit comment bodies pulled from merged PRs (#2482, #2870, #2871, #2873, #2890). Add a replay guard test that loads every fixture and asserts the expected verdict per PR (PASS for zero-finding walkthroughs, FAIL for rate-limit stubs). Add is_coderabbit_walkthrough regression guard that fails on origin/dev. RED proof (dev): ```text FAILED tests/scripts/test_check_bot_review_dev.py::TestReplayGuardDev::test_walkthrough_detector_recognises_real_bodies ... E AttributeError: module 'check_bot_review_dev' has no attribute 'is_coderabbit_walkthrough' ``` Removes-Intentionally: CODERABBIT_SCAFFOLDING_RE auto-summary alternation, classify() zero-finding loop
|
ⓘ Qodo reviews are paused because your trial has ended. Ask your workspace admin to add credits to resume reviews. Manage billing |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (9)
Included review availability: Your plan provides up to 4 included reviews per hour; 3 remain after this review. 📝 WalkthroughWalkthroughThe bot-review gate now classifies qualifying CodeRabbit walkthrough comments as real reviews. It detects failure notices separately, consolidates stub handling, and validates verdicts against live CodeRabbit comment fixtures. ChangesBot review gate
Estimated code review effort: 3 (Moderate) | ~30 minutes Merge Risk: ⚪ Minimal · up to Qualifying CodeRabbit walkthroughs are recognized while acknowledgements, failure notices, and rate-limit stubs remain rejected. No merge-blocking risk was identified. Sequence Diagram(s)sequenceDiagram
participant GitHubComments
participant BotReviewGate
participant WalkthroughDetector
participant Verdict
GitHubComments->>BotReviewGate: submit review-state and comment body
BotReviewGate->>WalkthroughDetector: classify walkthrough or stub markers
WalkthroughDetector-->>BotReviewGate: return real-review or stub result
BotReviewGate-->>Verdict: return PASS or EXIT_STUB
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 65.22% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 46 functions across 2 files. (7 skipped: 7 unsupported.)
✨ Finishing Touches 💡 2📝 Generate docstrings 💡
🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Code Review SummaryStatus: No Issues Found | Recommendation: Merge Files Reviewed (2 files)
Reviewed by step-3.7-flash:free · Input: 246.3K · Output: 25.7K · Cached: 619K |
jaylfc
added
the
gate-integrity-allow
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CARD TITLE (intent, not commit subject): FF #2900: bot-review-gate ack fix — real-body fixtures, replay guard, fenced red, trailer
Autonomous build of board card tsk-evhwml.
REVISION: built on
exec/tsk-qhqkdh(cut atb2cc40e9b87797cff491b687329a8d7edeb60681), not ondev. That branch'scommits are ancestors of this one. Verified by
git merge-base --is-ancestorbefore the PR was opened.
Replace synthesized walkthrough fixtures with genuine CodeRabbit comment
bodies pulled from merged PRs (#2482, #2870, #2871, #2873, #2890).
Add a replay guard test that loads every fixture and asserts the expected
verdict per PR (PASS for zero-finding walkthroughs, FAIL for rate-limit stubs).
Add is_coderabbit_walkthrough regression guard that fails on origin/dev.
RED proof (dev):
Removes-Intentionally: CODERABBIT_SCAFFOLDING_RE auto-summary alternation, classify() zero-finding loop
Files:
scripts/check_bot_review.py | 213 ++++++-----
tests/scripts/fixtures/coderabbit/2482.md | 414 +++++++++++++++++++++
tests/scripts/fixtures/coderabbit/2870.md | 105 ++++++
tests/scripts/fixtures/coderabbit/2871.md | 105 ++++++
tests/scripts/fixtures/coderabbit/2873.md | 208 +++++++++++
tests/scripts/fixtures/coderabbit/2890.md | 105 ++++++
tests/scripts/test_check_bot_review.py | 293 ++++++++++++---
9 files changed, 1298 insertions(+), 150 deletions(-)
Summary by CodeRabbit
Bug Fixes
Tests
The six test symbols below exercised the classification path this PR replaces; the dev-vs-PR replay over 20 merged PRs (with GH_TOKEN) showed 0 verdict flips, so their removal is intentional.
Removes-Intentionally: tests/scripts/test_check_bot_review.py:TestClassify.test_rate_limit_only_message_does_not_mention_scaffolding, tests/scripts/test_check_bot_review.py:TestClassify.test_scaffolding_only_message_does_not_mention_rate_limit, tests/scripts/test_check_bot_review.py:TestClassifyZeroFindingControls.test_findings_shape_alone_is_not_a_zero_finding_pass, tests/scripts/test_check_bot_review.py:TestDetectorIsolation.test_auto_summary_body_rejected, tests/scripts/test_check_bot_review.py:TestDetectorIsolation.test_neutering_auto_summary_loses_only_its_protection, tests/scripts/test_check_bot_review.py:TestIsRealItem.test_scaffolding_bodied_review_is_not_real_whatever_the_state