An AI pull request review workflow prevents an assigned human from starting a duplicate pass after agent comments land by giving the agent codebase context, explicit rules for thread ownership and merge gates, and a re-review.
TL;DR
When an agent comments before the assigned human reviewer begins, it creates a second review stream that needs explicit ownership. This walkthrough covers four decisions teams must make: who owns agent threads, which findings gate a merge, when new pushes trigger another agent pass, and how reviewers log disagreements without exhausting the review budget.
A pull request opens. The agent's summary lands before CI finishes scheduling, and four inline comments follow. A maintainer assigns the human reviewer an hour later; the reviewer scrolls past all five and starts a fresh pass. The author now has two review streams applying different standards to the same diff.
This walkthrough is for the developer on either side of that exchange: the PR author whose code the agent reviews and the reviewer sharing the queue. It covers the agent mechanics inside the PR workflow, from when the trigger fires to when the merge button becomes available. It does not define AI code review from first principles or rank tools.
Why AI Pull Request Review Is Hard to Get Right
A second reviewer with different failure modes requires explicit integration into the existing workflow. Cross-file logic is the sharpest example. No model detected more than 31% of human-flagged issues under any context configuration, per SWE-PRBench’s evaluation of eight models on 350 pull requests from active open-source repositories (arXiv:2603.26130v1, 2026). Contextual bugs, which require understanding how changed code interacts with unchanged code, made up 21.4% of the benchmark. An agent limited to the diff misses that category.
Timing creates a second review stream, and thread ownership creates a merge dependency. When an administrator enables GitHub's conversation-resolution requirement, one unresolved agent thread blocks merge exactly as a human thread does. A team that never decided who closes agent threads discovers this at merge time. Cosmos, a unified cloud agents platform, uses Environments to constrain execution, Experts to define agent behavior, and Sessions to preserve an auditable record across long-running engineering work.
Prerequisites
Before wiring an agent into branch protection, confirm the repository permissions and merge rules that govern its output. The same checks determine where the Expert can post findings and which human actions remain mandatory.
- Repository host and admin rights: GitHub or GitLab, with permission to edit protection settings. On GitHub, configuring "Require conversation resolution before merging" requires Admin access or a custom role with "edit repository rules." On GitLab, protected-branch settings require Maintainer or Owner.
- Agent installation: Install the review agent under a repository identity, either a GitHub App or bot user, or through a CI trigger. For Augment's Code Review, set the review trigger to Automatic or Manual Command. Disabled prevents reviews.
- Branch protection decisions: On GitHub, choose the required status checks that the contexts array represents in branch protection API version 2026-03-10. On GitLab, decide whether to enable only_allow_merge_if_pipeline_succeeds and only_allow_merge_if_all_discussions_are_resolved.
- Thread permissions: GitHub allows a PR author or a user with write access to resolve PR threads. Under the GitLab 18.6 Discussions API, the Developer role qualifies; Maintainer and Owner also qualify, as does authorship.
With those decisions recorded, the repository host can run the review lifecycle without creating an unexpected merge dependency.
How the Agent Moves Through a Pull Request
The workflow follows a PR from open to merge with the agent in the loop. AI PR review products differ mainly in trigger defaults and re-review scope, while their comments remain part of the host's normal review interface. In Cosmos, Triggers start Sessions for configured review events, while the repository's own protection rules determine whether the resulting findings become binding.
1. The Agent Fires on PR Open
The trigger fires when the PR opens or leaves draft state. Augment's Code Review Automatic trigger fires when a PR leaves draft; the Manual Command trigger waits until someone comments Auggie Review on the PR. In Cosmos, the trigger starts the Deep Reviewer Expert for the configured repository event. Deep Reviewer reads the full diff plus the codebase context the Context Engine retrieves. The Context Engine maps call graphs and cross-repository relationships across repositories containing hundreds of thousands of files. The first artifacts are a PR summary comment and inline comments on specific diff lines.
Silence on draft PRs is usually configured behavior. Draft exclusion is the default across the field. CodeRabbit ships reviews.auto_review.drafts: false in .coderabbit.yaml. GitHub Copilot's "Review draft pull requests" ruleset option stays off unless an administrator enables it. Greptile separately defaults triggerOnDrafts to false. To request feedback on a draft, comment auggie review on the draft PR or enable the relevant draft setting. Do not wait for ready-for-review and assume the agent failed.
2. Inline Comments and Suggested Fixes Land on the Diff
Comments attach to diff metadata. GitHub anchors each inline review comment to a commit_id, line, and position. When a later commit changes the anchored line, GitHub renders the thread as outdated. That anchoring explains why a heavy rebase can scatter agent threads across a PR's timeline.
Findings arrive as concrete fixes or advisory notes. Augment's Code Review offers a Fix in Augment flow that lets an author apply a proposed change directly. Advisory notes identify a caller or dependency that the diff affects without proposing an edit. Deep Reviewer and Pair Reviewer both post inline, and the Deep Reviewer workflow instructs the Expert to scan existing comments and their resolution state before posting so it does not repeat prior feedback. That review state gives the Expert context for the current pass. Authors should handle suggested fixes by applying or rejecting them. They should answer advisory notes in their threads because the next review pass can inspect that record.
3. Who Resolves a Thread and Who Can Dismiss One
Repository permissions control thread resolution and review dismissal, including comments that a Cosmos Expert posts. The two hosts divide those permissions as follows.
- GitHub thread resolution: The PR author can resolve any conversation, including one posted by an agent. Anyone else needs write access, and GitHub applies no separate rule for bot accounts.
- GitHub review dismissal: Write access suffices on unprotected branches. On protected branches where an administrator enables the "Restrict who can dismiss pull request reviews" rule, only an administrator or an allowlisted user, team, or app may dismiss the review. The allowlist can include GitHub Apps, users and teams.
- GitLab thread resolution: The Developer role qualifies; Maintainer and Owner also qualify, as does MR authorship.
- GitLab approval reset: Only bot users with project or group tokens can call the reset-approvals endpoint; human callers receive 401.
The agent holds less authority than its comment volume suggests. By default, GitHub Copilot leaves a "Comment" review, not "Approve" or "Request changes," so its reviews do not count toward required approvals or block merging unless a team configures it otherwise. Required approvals must come from people with write permission or designated code owners. The practical division is to let the author resolve addressed agent threads and reserve dismissal for the human reviewer, since dismissal removes a review's standing and does not merely close one conversation. Cosmos does not replace these host-level permission boundaries.
4. The Review Re-Runs on Re-Push
Re-review behavior is where tools diverge most. Augment's Code Review does not automatically fire on push; request a follow-up through the documented review commands. Comment auggie review or augment review on the PR. The command augmentcode review also requests a pass. CodeRabbit re-reviews automatically and incrementally by default with auto_incremental_review: true, which covers commits added since the last review. Its @coderabbitai review command starts an incremental pass, while @coderabbitai full review starts a complete pass that disregards prior comments. Copilot re-reviews only when an administrator enables "Review new pushes" in the ruleset. Teams should align the trigger choice with their push cadence, since not every push starts another Session.
Resolved threads create the most friction. Copilot may repeat previous comments during re-review even after an author resolved or downvoted them. CodeRabbit checks open threads during the next review and resolves those addressed by the changes. If duplicates reach a PR, switch Augment's trigger to Manual Command so re-review fires only on request, and keep incremental mode enabled where the product offers it. On Copilot, record a resolution reason. GitHub introduced resolution reasons on August 27, 2026. Use Addressed for completed changes; for findings not applied, use Won't fix or Incorrect. That record gives the next pass more context.
5. What Blocks Merge Versus What Is Advisory
Agent output remains advisory until branch protection makes it binding. GitHub provides two mechanisms. First, run the agent as a CI job. Add its check under "Require status checks to pass before merging"; the check must report successful or skipped before collaborators can merge, and a neutral result also permits merging. Second, enable "Require conversation resolution before merging", which blocks merge while any agent or human thread stays unresolved.
GitLab mirrors both mechanisms. "Pipelines must succeed" maps to only_allow_merge_if_pipeline_succeeds, while "All threads must be resolved" maps to only_allow_merge_if_all_discussions_are_resolved. External status checks can block merges on Ultimate when an administrator enables "Status checks must succeed." In Cosmos, Code Review returns findings that an engineer or another agent can resolve, but the repository configuration decides whether those findings affect the merge gate.
Everything outside those configured controls stays advisory. Copilot's reviews never block merge by design. Risk Analyzer, the Cosmos Expert that surfaces blast-radius and security-exposure signals on a PR, informs the gate rather than enforcing it; the team decides which signals become required checks. Start in advisory mode by running the agent without required checks for the first weeks. Measure which finding categories the team consistently acts on, then promote only those categories to blocking status.
When the Agent and the Human Reviewer Disagree
Repository permissions define the boundary for disagreements. Humans with the required access retain review and merge authority. Engineering leads should keep that checkpoint human-based on the cited acceptance and rejection rates. A July 2026 preprint on agentic code review (arXiv:2607.03316) analyzed over 31,000 review-feedback pairs across 239 repositories: CodeRabbit reviews were accepted 36.4% of the time and rejected 56.3% of the time. A separate August 2025 preprint (arXiv:2508.18771) covering 22,326 AI-generated comments across 178 repositories found that valid AI comments led to code changes at rates of 0.9% to 19.2%, against 60% for valid human review comments.
Reviewers can resolve the main disagreement patterns with explicit rules.
- Finding-level disagreements: When the human reviewer overrules an agent finding about style or convention, record the dismissal reason and encode that convention in the instruction file so the same disagreement does not recur. When the agent identifies a cross-file effect the human did not see, inspect the referenced call sites before dismissing it because call-graph retrieval reaches code outside the diff view.
- Gate-level disputes: Use the configured merge gate. Required checks determine whether automated validation passed, and required human approvals determine whether the change can merge.
Code Review returns findings that an engineer or another agent can resolve, while Cosmos preserves the review work in a Session and leaves the approval checkpoint with the repository's authorized reviewers. That division keeps the agent's evidence available without assigning it merge authority.
Tuning the False-Positive Budget
Noise is measurable: precision is the fraction of agent comments that match real issues, and everything below 100% is the budget the team spends resolving or dismissing comments that should not have fired. Teams can tune that budget by narrowing the signal scope and recording repository-specific conventions. Augment supports review guidelines through a configuration file. CodeRabbit documents its own review configuration separately. GitHub defines Copilot resolution behavior, and Greptile documents its draft trigger.
| Signal type | Typical noise pattern | Mitigation | Config lever |
|---|---|---|---|
| Style and nitpick comments | High comment volume on working code | Narrow scope to correctness issues | Augment review style: Precise; CodeRabbit reviews. profile: quiet |
| Convention false alarms | Agent flags patterns the team adopted deliberately | Encode standards in instruction files | .augment/code_review_guidelines.yaml; .github/copilot-instructions.md; CodeRabbit reviews.path_instructions |
| Repeated findings | Same comment reposted after resolution | Incremental re-review plus recorded resolutions | CodeRabbit auto_incremental_review: true; Copilot resolution reasons |
| Draft-stage churn | Reviews firing on work-in-progress commits | Keep draft exclusion defaults | CodeRabbit drafts: false; Greptile triggerOnDrafts: false |
| Severity inflation | Low-impact issues labeled high | Assign severity per rule | severity per rule in the guidelines YAML, using "high" or "medium"; "low" remains available |
Set expectations from measured precision. Augment's December 11, 2025, benchmark of seven code review tools ran on 50 pull requests from Sentry, Grafana, Cal.com, Discourse, and Keycloak. It reported 65% precision and 55% recall for Augment Code Review. The resulting F-score was 59%. Augment published the benchmark and expanded and corrected the golden comment set before scoring, so the result is first-party evidence with a checkable methodology. Within that corpus, about a third of comments did not match a golden issue, which is why the configuration layer matters.
The team should write the instruction file before changing a volume setting. Augment's review guidelines file supports areas scoped by glob, per-rule severity, and file_paths_to_ignore. The review process automatically discovers AGENTS.md and CLAUDE.md, so an engineer can convert a dismissed false positive into a standing rule for the next PR. Verbosity settings reduce volume uniformly through Augment's Precise style or CodeRabbit's profiles. Copilot's effort levels provide the same control. Guidelines suppress the specific comments that engineers have identified as wrong. For a Cosmos review Expert, those repository instructions constrain the findings produced during each review Session.
What to Do Next
The tradeoff is fixed: an agent reaches call sites and dependencies the diff view hides, and in exchange it adds comment volume the team must manage and tune. Teams that skip the tuning pay for the cross-file coverage in dismissal labor on every PR.
Run a two-week audit. Pull the last 20 agent threads on merged PRs and classify them by outcome, recording whether each resolved thread included a code change.
- Resolved: Record whether the resolution included a code change.
- Dismissed
Convert every dismissal reason into a rule in the guidelines file with an explicit severity, then set the re-review trigger to match the team's push cadence. Repeat the tally after two weeks. The team should observe fewer dismissed findings. If the count does not fall, remove the agent's checks from required status before reviewers start ignoring findings that matter.
Frequently Asked Questions
Related Reading
Written by
Paula Hingel
Paula writes about the patterns that make AI coding agents actually work — spec-driven development, multi-agent orchestration, and the context engineering layer most teams skip. Her guides draw on real build examples and focus on what changes when you move from a single AI assistant to a full agentic codebase.