The software factory needs a faster review loop: further optimizing the path from PR to merge

TL;DR

In an AI-native engineering organization, generating code is no longer the slowest part of shipping software. Review, verification, repair, and human decision-making become the bottlenecks, and ultimately define how much faster the organization can move. Standalone AI coding tools may deliver a 30–40% speedup, but achieving a step-change in throughput requires optimizing everything that happens after the code is written.

In our previous post about solving code review with Cosmos, we described a review system that helped our engineering organization increase code output by 3× while reducing median merge time and maintaining quality. We have since expanded that system beyond review into the full PR-to-merge loop: specialized agents handle mechanical work, verify correctness, and address feedback, while humans provide judgment and knowledge transfer, and always own the final merge decision.

This is loop engineering: improving the complete system that turns generated code into a verified, understood, merge-ready change rather than optimizing any single tool in isolation.

Why an “AI code review tool” is not enough

A conventional AI code review tool like CodeRabbit or Greptile analyzes a diff and posts comments. Useful, but review is only one step. The real drag is the chain of human handoffs:

• Reconstructing intent and architecture.
• Triaging low-risk changes.
• Applying review feedback.
• Fixing CI failures and merge conflicts.
• Verifying feature behavior end to end.
• Gathering enough evidence to ship safely.
• Re-reviewing after each push.

A PR-to-merge loop keeps specialized Experts working through review, repair, verification, and decision support until a human can merge with confidence. Each Expert in the fleet owns one or more of these handoffs, turning a fragmented sequence of manual tasks into a coordinated system. The goal is to optimize the entire loop: human time, cost, quality, and merge latency, not time to first comment.

1. Overview: engineering the complete loop

An AI-native code review loop: agents find and fix bugs, verify changes, and evaluate policy; humans resolve judgment calls and make the final merge decision.

Our original system separated risk analysis (Risk Analyzer), line-by-line correctness review (Deep Reviewer), and human-guided design review (Pair Review). The expanded system adds repair, runtime verification, and expanded auto-approval, reducing the human bottleneck further. For an overview of Cosmos and its configurable Experts, see our previous post about solving code review with Cosmos.

Experts and capabilities, not one general-purpose reviewer

The distinction between Deep Reviewer and Pair Reviewer is particularly important:

• Deep Reviewer asks: “Is there an objective bug in this implementation?” It runs autonomously and checks the PR against AGENTS.md or CLAUDE.md guidelines.
• Pair Reviewer asks: “Does this change make sense in the broader system, and which decisions require human judgment?” It runs interactively with the human. The human can also instruct it to monitor the PR after posting review comments and approve on their behalf once those comments are addressed.

PR Author or PR Fixer? Our previous post introduced PR Author, which writes a change and then shepherds its PR through review feedback and failures. PR Fixer performs the same repair work but can start from any existing human- or agent-authored PR. This post focuses on PR Fixer because it works regardless of how the PR was created.

The Review Dashboard brings Expert status, reviewed commits, evidence, and available actions into one view.

2. Designing the human-in-the-loop

Human review and verification are becoming the scarce resources in AI-native engineering organizations. The goal is not to remove humans indiscriminately. It is to spend human attention only where it has the highest leverage.

Why humans remain in the loop

Agents can perform a large share of mechanical analysis and execution, but they do not have complete business and organizational context. Humans remain essential for:

• Judgment calls: Should this logic live in the frontend or backend? Is this tradeoff appropriate for the product? Is this risk acceptable now?
• Knowledge transfer: Review is one of the ways engineers build a shared understanding of architecture and product behavior.
• Ownership and accountability: Agents do not own the software after it ships; human developers and engineering organizations do. A human therefore always makes the final decision and clicks Merge. None of these experts merges a PR.

The design target is therefore:

Agents do the mechanical work. Humans make the consequential decisions.

The author and reviewer workflow: before and after

The economic intuition of using multiple experts

Tokens that measurably reduce human time in the review bottleneck are worth the investment. When Experts turn hours of review and PR shepherding into minutes, they free scarce engineering judgment for the consequential decisions only humans can make, while also helping features reach customers faster.

One of Augment’s core commitments is helping organizations optimize costs. That means optimizing total cost per task: human effort plus token cost. It does not mean minimizing token usage at the expense of successful outcomes. A single Expert burdened with six responsibilities will do a substandard job of each and require more human intervention. Six dedicated Experts can each focus on a different part of PR review, produce higher-quality review artifacts, and drive more of the process autonomously.

3. Cost and quality: optimize cost per successful outcome

The cheapest model by token price is often a false economy. A missed defect, bad repair, or retry can cost more than getting the task right once. We benchmark representative work and choose the lowest-cost model that clears each Expert’s quality bar, the same cost-per-success principle we use across Cosmos.

Today, we use GPT-5.6 Sol for judgment-heavy work such as Risk analysis, Deep and Pair review, and code repair. Bounded, mechanically verifiable tasks such as dashboard aggregation and merge-conflict monitoring run on GPT-5.6 Luna. Models with longer cache TTLs also favor long-running agents because cached input is typically discounted at 90%.

4. A configurable approval path with cosmos approve

The Risk Analyzer has always been able to approve intrinsically low-risk changes under a conservative policy. We now support a second, disabled-by-default approval mode for other changes.

The PR author can comment cosmos approve to request an evaluation against an approval policy. A human still owns and performs the final merge.

Organizations define their own approval policy. Our internal policy checks:

1. Ownership: The requester is the PR author, and the author is an effective CODEOWNER for every modified file.
2. Current-head review: No unresolved Deep Reviewer findings, Pair Reviewer blockers, or unaddressed human comments.
3. No contradictory runtime evidence: The Verifier has not reported an unaddressed defect against the current commit.

5. Customizability is part of the architecture

We designed the fleet so teams can customize the loop with Cosmos Advisor.

• Fleet: Add, remove, or replace experts; choose the model and prompt for each responsibility.
• Operation: Configure triggers, tools, integrations, and verification environments.
• Control: Restrict who can invoke actions and which credentials, repositories, and systems each expert can access.
• Policy: Define approval rules, required checks, CODEOWNER requirements, and acceptable evidence.

The operating model for AI-native PR Review

The lesson from our first version was that code review could not scale by asking humans to read AI-generated code faster. The lesson from this version is broader: no individual review agent can optimize the full path to merge.

A high-functioning PR-to-merge loop needs:

• Agent Specialization: Separate risk, correctness, design judgment, runtime verification, and repair.
• Minimize human checkpoints: Pull humans in for consequential judgment and knowledge transfer only.
• Evidence: Give reviewers inspectable proof rather than unsupported verdicts.
• Repair loops: Let findings flow back into implementation without waiting for manual intervention.
• Observability: Make the state of the entire fleet legible in one place.
• Cost discipline: Use the cheapest model that clears the quality bar for each responsibility.
• Configurability: Customize the loop to each organization’s unique requirements.
• Human ownership: Keep the final merge decision with the people accountable for the software.

This is loop engineering applied to the PR lifecycle: optimize the system that produces a verified, understood, merge-ready change, not the amount of output produced by any single agent.

Build your own PR-to-merge loop

Cosmos gives engineering teams the shared context, runtime controls, integrations, and human checkpoints to run agents across review, verification, repair, and the rest of the software lifecycle.

Try Cosmos