Claude Code Review changes the outcome on a pull request only when someone owns the findings and a separate gate enforces them, because the check run it writes is advisory and never blocks a merge.
TL;DR
Automated review now reads the whole repository and verifies its own findings, so the weak link moved downstream. Teams enable Claude Code Review, get accurate findings, and then discover the check run was never a gate and nobody owns the queue. REVIEW.md is the tuning surface most teams skip.
An automated review flags a null dereference three files away from the diff. The check run finishes neutral, the author moves to the next ticket, and the pull request merges anyway. Months later the same bug lands in a different service, and the review flags it again to the same effect.
This guide is for engineers, tech leads and platform engineers evaluating Claude Code Review for a GitHub pull request workflow. It covers what the agents inspect, what starts a run and what it costs, and how findings reach the author. Then how CLAUDE.md and REVIEW.md change the output, where the work gets handed back, and how it scored on the December 11, 2025 benchmark. Custom GitHub Actions pipelines and GitLab CI/CD setups are outside scope; the managed GitHub App is the subject.
Why Review Findings Now Outrun the Teams Reading Them
The bottleneck in automated review used to be detection. A linter saw one file, a static analyzer one call graph, and anything needing the surrounding code went to a human. That constraint is gone: a review agent reads a change against the whole repository, verifies its candidates before posting, and returns findings a senior engineer would have taken an afternoon to reach.
What did not change is who acts on them. NIST's Secure Software Development Framework, published as Special Publication 800-218, asks organizations under practice PW.7.1 to decide whether code review, code analysis, or both belong in a workflow. Practice PW.7.2 then asks them to record and triage every discovered issue in the development team's workflow or issue tracking system. The second half is the one teams skip, and a faster, cheaper reviewer only makes an unowned queue grow faster.
What Claude Code Review Inspects
Anthropic's Code Review documentation, read on September 3, 2026, describes a fleet of specialized agents examining a change "in the context of your full codebase," each looking for a different class of issue. It does not say which classes each agent owns, only that they run in parallel and hunt separately for logic errors, security vulnerabilities, broken edge cases and regressions.
What happens next is the part worth understanding. Candidates go to a verification step that checks each against actual code behavior and drops what fails, which is why a plausible finding inferred from a function's name never reaches the pull request. Later stages deduplicate and rank the survivors, and the reporting stage posts them inline with a summary in the review body. The default target is correctness: bugs that would break production, not formatting preferences or test coverage. Guidance files widen that scope and nothing narrows it.
The Pre-existing severity tier settles how far it looks. The same page defines that tier as a bug already in the codebase but not introduced by this pull request, and a diff-scoped reviewer could not produce the finding at all. The cost of that reach shows up on legacy code, where a correct Pre-existing finding is still not the author's problem this week; REVIEW.md skip rules and severity definitions exist for that case.
Availability, Triggers, and What a Review Costs
Anthropic's Code Review page lists this as a research preview for Team and Enterprise subscriptions, closed to organizations with Zero Data Retention (ZDR) enabled, which for most teams settles whether the rest applies. An Owner or Primary Owner enables it and picks the repositories it may access, installing the Claude GitHub App. Running Claude in a team's own infrastructure is a different product: GitHub Actions, GitLab CI/CD, or GitHub Enterprise Server.
Owners set a Review Behavior mode per repository: once after pull request creation, after every push, or manual. The middle option is the expensive one and the most useful one, since it auto-resolves threads as later pushes fix what was flagged, but on a pull request that takes fifteen pushes it bills fifteen times. Whichever mode is set, a top-level comment overrides it: @claude review and @claude review once run once without subscribing to later pushes, @claude review always runs once and subscribes. Fork pull requests have no override at all, since only a comment starts one and re-running the check does nothing.
A review takes 20 minutes on average and costs $15 to $25, scaling with pull request size, codebase complexity, and how many candidates need verification. Anthropic bills it on token usage through usage credits, outside the plan's included usage, which is why the trigger mode above is the only cost lever most teams will touch.
Teams outside those plans still have a local path. The /code-review command reviews a diff in the terminal without the GitHub App, covering the branch's commits ahead of upstream plus uncommitted changes, and runs as a background subagent with its own context window. It is not the managed service in miniature: it follows CLAUDE.md but never reads REVIEW.md, so none of the tuning below reaches it, and /code-review ultra escalates to a cloud review closed to ZDR organizations.
Severity Levels and How Findings Reach the Author
Three severities carry every finding, and the naming matters before anyone parses the output, because the check run writes Important under the key normal.
| Severity | Meaning |
|---|---|
| Important | A bug that should be fixed before merging |
| Nit | A minor issue, worth fixing but not blocking |
| Pre-existing | A bug in the codebase that was not introduced by the request |
The same finding is written to three surfaces, which matters because GitHub silently drops one of them whenever a line moves.
| Surface | Behavior |
|---|---|
| Inline comments | Land on the specific lines, each with a collapsible extended-reasoning section and pre-attached thumbs-up and thumbs-down buttons whose ratings feed Anthropic's tuning |
| Check run Details | Carries a severity-sorted table of Severity, File:Line, and Issue |
| Files changed annotations | Render red for Important, yellow for Nit, and gray for Pre-existing |
The table and the annotations are written independently of the inline comments, so both survive when GitHub rejects a comment on a moved line. Replying does not prompt Claude to respond or change anything: the loop is to fix the code and push, and on a subscribed pull request the next run resolves the thread once the issue is gone.
Tuning Reviews With CLAUDE.md and REVIEW.md
CLAUDE.md and REVIEW.md both change what a review reports, and they are not interchangeable: one is the project context every Claude Code task already reads, the other exists only for reviews and reaches further in.
CLAUDE.md as Project Context
Claude reads CLAUDE.md as project context and flags a newly introduced violation of a rule stated there as a Nit, at every level of the directory hierarchy, so a rule in services/billing/CLAUDE.md binds only files under that path. It runs both ways: a pull request that makes a CLAUDE.md statement outdated, such as changing a documented default, gets flagged too. The file grows because every other Claude Code task reads it, which is why review-specific rules belong in REVIEW.md.
REVIEW.md as Review Instructions
The review pipeline gives REVIEW.md directly to the finding and verification agents, and the ranking and reporting agents consult it before settling severity and writing the review.
- Severity definitions: The default calibration is tuned for production code, the wrong bar for a docs repo or a prototype. Naming a missing tenant check as Important keeps the ranking agents from filing it as a Nit.
- Nit cap: Prose and config files can be polished forever. Capped at five, the rest arrive as a count in the summary.
- Skip rules: Generated protobuf output, lockfiles and vendored dependencies spend verification budget on findings nobody will act on. This is also where a team stops duplicating continuous integration (CI), since formatting and import order belong to a linter.
- Always-check rules: "Every new database migration needs a matching down migration" puts that check in front of every reviewer on every branch. Rules this specific land more reliably here than in a long CLAUDE.md, because REVIEW.md reaches the finding and verification agents directly.
- Verification bar: Behavior claims that need a file and line citation are behavior claims no agent can infer from a function's name.
- Re-review convergence: New Nits suppressed after round one, Important findings only from then on. Without that line a one-line fix reaches round seven on style alone.
- Summary shape: A one-line tally at the top, two factual and four style, gives the author the shape of the work before the details.
The pipeline reads REVIEW.md as plain text and does not expand @ import syntax or follow file references, so every rule has to live in the file itself. Length then works against it, because a long file dilutes the rules that matter most.
Where Automated Review Stops
Anthropic documents the boundary. Findings are tagged by severity and do not approve or block the pull request, and the check run always completes with a neutral conclusion so branch protection never blocks a merge on it. That is not a soft no: GitHub's required-check documentation lists neutral alongside success and skipped as a conclusion that counts as passing.
Anyone who wants a gate builds it, and the raw material is there. The last line of the check run's Details text is a machine-readable comment shaped like {"normal": 2, "nit": 1, "pre_existing": 0}. A CI job reads normal with gh and jq, fails above zero, and becomes the required check while Code Review stays neutral.
After the review posts, nobody dispatches the fix, nothing authors a patch, and nothing runs the patched code against a live environment. The thumbs ratings go to Anthropic to tune the reviewer; they do not come back as memory on the next pull request in the same repository. Cosmos, Augment Code's unified cloud agents platform, is available on all paid plans and puts Experts on the downstream review steps. Both sides read the repository; the difference is what happens once a finding exists.
- Reviewing: Cosmos Deep Reviewer covers the same ground as an advisory reviewer, line by line against AGENTS.md and CLAUDE.md. The other two triage: Risk Analyzer can auto-approve low-risk changes with a short justification and flag the rest, and Pair Reviewer takes the flagged ones interactively, where design intent or rollout matters more than scanning every line.
- Acting: PR Fixer answers findings, diagnoses CI failures and resolves merge conflicts it can read unambiguously. PR Author works the other direction, ticket to reviewed pull request. Verifier runs the change in a live environment and reports what it saw, what it could not see, and the evidence for each.
No Cosmos Expert merges anything: PR Fixer applies fixes and stops, Verifier reports evidence and stops, and a human still clicks the button. What changes is what survives the merge. Cosmos review memory captures the human comments, the reactions to agent findings and the outcome, then curates them into repository guidance the review Experts load next time, which is the one thing an advisory reviewer structurally cannot do.
None of which makes the trade obvious. Cosmos is infrastructure a team runs. Claude Code Review is a GitHub App with nothing to host, a verification pass that discards false positives before the author sees them, and a tuning surface in REVIEW.md that reaches unusually far into the pipeline.
Accuracy and Tool Comparison
The figures below come from a seven-tool review benchmark published here on December 11, 2025, scored on 50 pull requests from Sentry, Grafana, Cal.com, Discourse and Keycloak. Read them as a first-party result: Augment Code Review is one of the seven, and the golden comment set was expanded and corrected in house before any tool was scored against it. That meant reviewing each pull request by hand, validating every addition against tool outputs, and adjusting severity so trivial suggestions could not move a score. The corrected data and scripts are open source. This is the only benchmark here that includes Claude Code, sorted by F-score:
| Tool | Precision (%) | Recall (%) | F-score (%) |
|---|---|---|---|
| Augment Code Review | 65 | 55 | 59 |
| Cursor Bugbot | 60 | 41 | 49 |
| Greptile | 45 | 45 | 45 |
| Codex Code Review | 68 | 29 | 41 |
| CodeRabbit | 36 | 43 | 39 |
| Claude Code | 23 | 51 | 31 |
| GitHub Copilot | 20 | 34 | 25 |
Augment Code Review tops that set at 59%, which is the result a reader should discount for exactly the reason above and check against their own repositories.
Change the metric and Claude Code moves from sixth of seven to second: sixth on precision, second on recall-first review. That is a reviewer that found most of the golden issues and, in that build, buried them in findings the golden set did not accept. Codex Code Review is the mirror image, first on precision and seventh on recall. The post is dated December 11, 2025, carries a June 18, 2026 update stamp, and reports Claude Code's recall in the present tense, so the page does not settle which build the number describes. Either way the move is the same: treat the table as a floor and re-test on a team's own repositories.
Where Teams Get This Wrong
The recurring failures show up after enablement, once reviews are running and someone has to decide what happens to a finding.
- Expecting the check run to block a merge: It never will. Branch protection waves a pull request through with three Important findings open, so the gate has to be a separate CI job reading the severity counts, and that job has to be the required check.
- Leaving findings unowned: The queue ages until the pull requests merge around it. Name the owner before switching review on, whether that is the author, a rotation, or a Cosmos Expert.
- Skipping REVIEW.md and blaming the model: Generated code and lockfiles stay in scope, the default calibration decides what counts as Important, and the noise that follows gets read as a model problem. Write the file before judging the output.
- Running after-every-push on a busy repository: Every push is another $15 to $25 against usage credits, so a twelve-push pull request costs twelve reviews. Manual or once-after-creation, on anything with real traffic.
- Assuming preview access covers the organization: Eligibility is set per organization and per repository, so a team on the wrong plan tier, or with ZDR on, never sees a review at all. Confirm both, then check the repository list before announcing anything.
Only the second of those is a judgment call. The other four are settings nobody read.
How to Put Code Review to Work
Confirm eligibility first: the feature needs Team or Enterprise with ZDR off. Ask an Owner to enable Code Review, install the GitHub App, and add one repository in manual mode, ideally a small service with low pull request volume.
Run the first reviews by hand with @claude review on pull requests you already understand, and read the Pre-existing findings separately from the Important ones. On a legacy service that first batch runs long, and none of it is the author's doing. Log those as tickets for whoever owns the module and keep them out of the pull request.
Write REVIEW.md before widening to a second repository: what Important means for this codebase, a Nit cap, skip rules for the files CI already checks, and a verification bar requiring a file and line citation. Keep that first pass short. Skip rules for lockfiles and generated code, plus one severity definition tied to a bug class that has actually shipped, cover most of the noise.
Decide who owns acting on findings, and write it down. If nobody wants the queue, the honest options are a Cosmos Session started by a GitHub Trigger, or leaving review advisory and accepting that findings will sometimes merge unaddressed. If you want a gate, add the gh and jq job before the team learns to ignore neutral checks.
What to Do Next
Advisory review is cheap to adopt and changes nothing on its own. Acting on findings automatically, whether through a gate or a Cosmos Expert, is where both the cost and the caution live. Pick one repository this week, enable Claude Code Review in manual mode, and name an owner for Important findings before the first review runs.
Frequently Asked Questions
Related Reading
- 8 Best AI Coding Assistants by Job [Updated August 2026]
- 10 Open Source AI Code Review Tools Tested on a 450K-File Monorepo [2026 Rankings]
- 5 Best Agentic Development Environments for Enterprise Teams in 2026
- 6 Best Spec-Driven Development Tools for AI Coding in 2026
- 9 Open-Source Agent Orchestrators for AI Coding (2026)
Written by
Molisha Shah
Molisha is an early GTM and Customer Champion at Augment Code, where she focuses on helping developers understand and adopt modern AI coding practices. She writes about clean code principles, agentic development environments, and how teams are restructuring their workflows around AI agents. She holds a degree in Business and Cognitive Science from UC Berkeley.