The Coordinator-Implementor-Verifier approach is a three-role test-authoring architecture that separates coverage planning, executable test generation, and independent correctness verification.
TL;DR
CIV test authoring assigns coverage planning, executable generation, and correctness verification to separate roles. Single-agent test generation poses an oracle risk because the same model both writes and certifies the test. Playwright v1.56's native Planner/Generator/Healer agents show this pattern in Node.js/TypeScript workflows.
The developer's frustration is familiar: an AI agent can produce a passing test in seconds, but the reviewer still has to ask whether the test validates intended behavior or only mirrors the current implementation. The reviewer must trace the requirement, inspect assertions, and decide whether a green test increases coverage or preserves a bug. That review burden compounds once generated tests enter a production suite, because invalid tests become regression gates that protect the wrong behavior.
Research on LLM-based test generation documents invalidly generated unit tests as a hallucination failure mode. That makes single-agent self-certification a production-suite review risk. Playwright v1.56 shipped this shape natively through Planner, Generator, and Healer agents. This guide maps CIV to test authoring, explains where Playwright's implementation fits, compares the multi-agent evidence against single-agent generation, and outlines adoption tradeoffs.
One of those tradeoffs is whether to build that coordination layer at all. Augment Cosmos, the unified cloud agents platform, already exposes this same three-role split as named agent experts, Coordinator-style planning, Implementor execution, and Verifier-style review, running on a shared workflow layer included in every paid plan rather than sold as a separate add-on. Teams evaluating code-generation quality on that platform can measure Implementor-style output against 70.6% SWE-bench Verified accuracy.
That dependency question becomes practical before suite admission: reviewers need to know which files a verifier should inspect before tests become regression gates.
The New Code Review Workflow for AI-Native Engineering Teams
See how leading teams keep code review fast and rigorous as AI writes more of the code.
Why Single-Agent Test Generation Needs Review Gates
That hallucination failure mode has a specific shape in practice. Researchers describe a "circularity of error" in which existing LLM-based test generation uses the implementation under test as the ground-truth oracle. Models generate tests that match the current code logic rather than the intended specifications, so those tests cannot detect faulty behavior.
CIV turns that risk into reviewable acceptance gates before a test enters the suite.
- Coverage planning gate: The Coordinator interprets requirements and translates them into scenarios before test code is written.
- Executable generation gate: The Implementor converts approved scenarios into runnable tests with build and pass/fail filtering.
- Correctness verification gate: The Verifier checks expected behavior against requirements before suite admission.
These gates keep generating tests downstream of the requirements, rather than letting current implementation behavior become the only oracle.
Agent design research explains why the roles need to be separated. As one analysis puts it: "The planner pushes for clarity and completeness, the executor pushes for pragmatic implementation, and the critic pushes for correctness and standards compliance." Test authoring needs that counterbalance because a passing test says nothing about whether it tests the right thing.
When teams move from one-off use of AI testing tools to a continuous, agent-authored suite, the three-role separation provides reviewers with explicit checkpoints for coverage, execution, and correctness.
Coordinator-Implementor-Verifier Uses Planned Subtasks and Independent Review
The Coordinator-Implementor-Verifier pattern uses three agent roles. A Coordinator decomposes a task into a dependency-ordered plan. Implementors execute scoped subtasks in isolated contexts. A Verifier validates each output against the original specification. CIV derives from the orchestrator-worker pattern, which Anthropic defines as a workflow in which orchestrators direct agents to complete subtasks toward a broader goal.
Academic frameworks explicitly formalize all three roles. VeriMAP defines a Planner that generates instructions and coordinates subtasks across agents. Its Executor solves an assigned subtask using planner-generated instructions and context. Its Verifier produces a binary success/fail evaluation with detailed feedback.
| Role | Synonyms in Research Literature |
|---|---|
| Coordinator/Orchestrator | Supervisor, Planner, Lead Agent, Manager Agent |
| Implementor/Worker | Executor, Sub-agent, Specialist Agent, Tool Agent |
| Verifier/Critic | Evaluator, Improver, Reflexion Actor-Critic, Verifier Agent |
Across the implementations discussed here, the execution sequence follows the same mechanism. The planner generates an execution DAG with pre-declared success criteria and acceptance gates. Writers and executors perform the work. Critics evaluate outputs against the success criteria.
Playwright v1.56 Uses Planner, Generator, and Healer Agents to Split Test Creation from Repair
Playwright v1.56 splits test authoring into Planner, Generator, and Healer agent definitions. For Node.js/TypeScript teams, that split separates scenario planning, executable test creation, and post-failure repair.
Playwright officially describes Test Agents as tools designed "to guide LLMs through the core process of building a Playwright test." Per the official documentation, the agents "can be used independently, sequentially, or as the chained calls in the agentic loop. Using them sequentially will produce test coverage for your product."
Each agent maps to a CIV role.
- 🎭 Planner "explores your app and produces a test plan for one or many scenarios and user flows." It takes a seed test (
seed.spec.ts) as input and outputs Markdown files inspecs/containing an application overview and numbered test scenarios with steps and expected results. - 🎭 Generator "uses the Markdown plan to produce executable Playwright Tests. It verifies selectors and assertions live as it performs the scenarios." Its output is executable test files in
tests/, "aligned one-to-one with specs wherever feasible." - 🎭 Healer activates on failing tests, intended to "automatically repair failing tests by inspecting the UI and updating locators."
Playwright implements agent definitions on a specific substrate: "Under the hood, agent definitions are collections of instructions and MCP tools." Playwright provides the agent definitions and recommends regenerating them whenever Playwright receives an update. Agents initialize through npx playwright init-agents --loop=<target>, with supported targets including vscode, claude, codex, and opencode.
One detail matters for Test Architects mapping Playwright to CIV. Playwright's Healer performs post-execution repair. It fires after a test fails, updates locators, and re-runs. A CIV Verifier adds a separate gate by validating the correctness of tests before execution. Therefore, a complete verification loop using Playwright requires an external correctness check.
External review tooling fits at that boundary. For teams adding review to the loop, Augment Cosmos's Deep Code Review expert can support AI code review. Augment evaluated tools against golden comments using precision, recall, and F-score and reports a 59% F-score in its public benchmark of 50 real PRs.
Enterprise planning constraints also apply. The Playwright Test Agents feature is Node.js/TypeScript only, unavailable to teams using Java bindings or Python. And v1.56.0 shipped with a known healer bug involving mismatched tool names.
How CIV Stages Become Acceptance Gates
Each CIV stage assigns a different role to coverage interpretation, code construction, and oracle review.
| Acceptance Gate | CIV Role | Primary Input | Output | Acceptance Question | Failure Mode Controlled |
|---|---|---|---|---|---|
| Coverage planning | Coordinator | Requirements, code structure, and risk-weighted gaps | Scenarios before the test code exists | Do scenarios reflect intended behavior before generation starts? | Coverage work starts from the current implementation behavior alone |
| Executable generation | Implementor | Coordinator-approved coverage plans | Runnable tests with build and pass/fail filtering | Do generated tests build, run, and match the approved scenario? | Non-building or non-passing tests enter review |
| Correctness verification | Verifier | Generated tests and requirements | Correctness review before suite admission | Do assertions encode expected behavior rather than generated behavior? | The same model writes and certifies the test |
| Playwright implementation | Planner, Generator, and Healer | Seed tests, Markdown plans, and failing tests | Specs, tests, and repaired locators | Does the built-in flow cover planning, generation, and repair? | Healer repair is mistaken for pre-execution correctness verification |
| Adoption boundary | QA and SDET workflow owners | Complex work with verifiable subtasks | Role-specific acceptance gates | Does orchestration cost buy independent planning or verification value? | Multi-agent handoffs add cost without review benefit |
Coverage Planning Uses a Coordinator to Turn Requirements and Code Structure into Test Scenarios
Coverage planning uses a Coordinator agent to analyze requirements, code structure, and risk-weighted gaps before any test code exists. The Coordinator produces scenarios that constrain later generation work.
NVIDIA's HEPH framework uses LLM agents "from document traceability to code generation," identifying relevant Software Architecture Document and Interface Control Document fragments before generating test specifications. CovQValue treats a program's branch structure as an unknown environment and selects the most informative test plan by estimating expected branch gain.
In the CANDOR multi-agent test generation framework, Planner removal is the boundary condition in the reported ablation. Without the dedicated Planner agent, line coverage, branch coverage, and mutation score moved downward. The CANDOR paper describes the Planner as analyzing coverage gaps and proposing new test cases accordingly.
Test Generation Uses an Implementor to Convert Coverage Plans into Executable Tests
The Implementor converts Coordinator-approved coverage plans into executable tests. Its work includes code generation, build validation, pass/fail filtering, and, in Playwright v1.56, live selector-and-assertion checks. Keeping this Implementor boundary explicit ensures generated tests remain downstream of the Coordinator plan.
The common code-to-test pattern starts with the source code of the focal method and its class context. It outputs a complete unit test case across context analysis, test prefix generation, and oracle inference. Meta's TestGen-LLM applies "Assured LLM-based Software Engineering," filtering candidate tests through hard gates. It drops non-building tests, discards non-passing tests, and removes tests that don't increase coverage.
For teams building test automation tools, the generator stage adds a live selector-and-assertion validation gate during Playwright test creation. The boundary is Playwright v1.56's Node.js/TypeScript workflow. The Generator verifies selectors and assertions during generation, checking those browser-facing test details before they reach the test file.
The three CIV roles use different evidence checkpoints before generated tests reach suite admission.
| Checkpoint Dimension | Coordinator | Implementor | Verifier |
|---|---|---|---|
| Primary evidence | Requirements, code structure, and risk-weighted gaps | Coordinator-approved coverage plans | Generated tests and requirements |
| Main artifact | Scenarios before the test code exists | Runnable tests with build and pass/fail filtering | Correctness review before suite admission |
| Validation signal | Scenario coverage against intended behavior | Build validation, pass/fail filtering, and live selector-and-assertion checks | Assertion review against expected behavior |
| Failure controlled | Coverage starts from the current implementation behavior alone | Non-building or non-passing tests enter review | The same model writes and certifies the test |
| Playwright boundary | Planner produces Markdown plans | Generator produces executable Playwright Tests | Healer repairs failures but does not provide pre-execution correctness verification |
This checkpoint split keeps generation downstream of planning and keeps oracle review separate from executable test construction.
Verification Uses a Verifier to Separate Oracle Review from Test Generation
Verification uses a separate Verifier agent to review expected behavior against requirements. Oracle generation "determines the expected behavior of the software and encodes it into assertions... Oracle generation is the final and perhaps most challenging phase." A single agent tends to certify the behavior it just generated, so the verifier needs a separate role and a separate reference point.
CANDOR's two-phase oracle correction demonstrates the value of separation. CANDOR generates tentative oracles from source code. Then, a Requirement Engineer agent, plus a panel discussion, corrects them based on natural language descriptions.
CANDOR's reported ablations identify the Requirement Engineer agent and panel discussion as correctness gates. When either is removed, oracle correctness decreases. The full multi-agent pipeline uses oracle correctness as its comparison point against the state-of-the-art prompt-based test generator LLM-Empirical.
Context Engine supports architectural-level understanding, processes entire codebases across 400,000+ files, and uses semantic dependency graph analysis. That repository view lets verifier agents inspect cross-file dependencies before suite admission.
Multi-Agent CIV Separates Opposing Test-Authoring Roles Before Suite Admission
CANDOR applies multi-agent separation by assigning the interpretation of requirements, test construction, and oracle correction to different roles. That separation avoids asking one model to perform and approve the same work.
| Evaluation Area | Multi-Agent Pattern | Single-Agent Risk |
|---|---|---|
| Oracle correctness | Requirement-aware review corrects generated oracles | Generated tests can mirror the implementation under test |
| SWE-bench-style resolution | Specialized agents decompose and validate code changes | One model must plan, edit, and judge its own output |
| Decision determinism | Role separation creates clearer handoffs and acceptance gates | Recommendations can become vague or inconsistent |
| Multi-file signal detection | Specialized agents isolate cross-file dependencies | Single-path reasoning can miss the distributed context |
| Research evaluation | Multi-agent workflows benefit when outputs are verifiable | Single-agent workflows depend on self-certification |
Context isolation is the first mechanism for authoring CIV tests. Each Implementor receives a scoped task while the Coordinator maintains the broader plan.
A single agent routes all reasoning, planning, and tool use through a single shared reasoning path, and research notes that "a single agent can easily become a bottleneck under high concurrency, and its generalization capability is fundamentally constrained by the capacity of the underlying LLM." In CIV, each Implementor operates in an isolated Environment with a dedicated context while the Coordinator maintains architectural awareness.
Augment Cosmos's Auggie agent supports complex multi-file Implementor work when teams need scoped generation rather than single-file edits. Auggie plans multi-step tasks autonomously and executes scoped work without pulling the Coordinator off the broader plan.
Layered verification is the second mechanism for authoring CIV tests. Verifier gates combine LLM-based reasoning, static analysis, dynamic testing, and formal methods, as no single signal can validate every generated test. Anthropic makes a related point about agentic coding more generally: code solutions are well-suited to agentic verification because they can be checked with automated tests and their output quality can be measured objectively.
| CIV Mechanism | Coordinator Responsibility | Implementor Boundary | Verifier Signal | Benefit | Orchestration Cost |
|---|---|---|---|---|---|
| Context isolation | Maintain the broader plan | Execute a scoped task in a dedicated context | Check output against the plan | Limits shared-path reasoning | Requires state management across roles |
| Layered verification | Define success criteria | Produce test output for evaluation | Combine reasoning, static analysis, dynamic testing, and formal methods | Avoids relying on one validation signal | Adds review gates and tool calls |
| Decision determinism | Separate planning from validation | Follow scoped subtasks | Produce actionable correctness feedback | Creates clearer handoffs and acceptance gates | Adds handoff tracking |
| Multi-file signal detection | Maintain architectural awareness | Work within isolated dependencies | Inspect cross-file behavior before admission | Reduces the missed distributed context | Requires context coordination |
| Adoption boundary | Select work that decomposes into verifiable subtasks | Avoid overusing agents for simple, scoped work | Gate only where independent review adds value | Fits full test authoring | Weak fit for single-file work |
Incident-response decision-quality findings give QA leaders a bounded example for applying multi-agent systems. In that incident-response setting, the comparison emphasized the quality of recommendations and determinism rather than latency alone.
Single-agent systems produced vague, inconsistent recommendations, while multi-agent systems produced deterministic, actionable guidance. Separated planning and validation roles made task ownership clearer and recommendations more actionable in that setting.
Multi-Agent Test Authoring Adds Orchestration Cost Through Extra Agent Handoffs
Multi-agent test authoring increases orchestration cost through prompts, tool calls, state management, and review gates. Orchestration overhead is a major economic constraint in the adoption model because each additional role introduces another handoff that teams must track. Teams need to justify the overhead with work that can be decomposed into independent subtasks with verifiable outputs.
Coordination failure is a named failure category for multi-agent systems. A systematic study identified reasoning-action mismatches and wrong assumptions as recurring sources of failure. The conclusion: "Many MAS failures arise from the challenges in organizational design and agent coordination rather than the limitations of individual agents."
Microsoft's Azure SRE team built toward multi-agent specialization, then reversed course after finding that handoffs hurt reliability.
The pattern is a weak fit for single-file or simple scoped work, where coordination overhead does not add an independent planning or verification benefit. The CIV advantage appears when tasks decompose into independent subtasks with verifiable outputs, which is precisely the shape of full test authoring across a coverage plan.
Augment Cosmos's Context Engine gives onboarding teams a repository-wide view for verifier-first workflows. Context Engine shortens onboarding by automatically transferring repository conventions across contributors.
For SDETs, the practical work shifts toward defining agent handoffs, coverage priorities, and review gates. Teams coordinating complex, multi-file test authoring with Augment Cosmos's Context Engine move faster because the dependency graph exposes scoped dependencies before Implementors begin work.
Convergent Testing Tools Use Plan-Implement-Verify Loops to Shift Work from Scripts to Coverage Plans
The testing and automation systems discussed here vary in how explicitly they structure test authoring around planning, implementation, and verification. Playwright, Magnitude, Teradata code assistant, UiPath, and TestQuality all solve the same test-authoring problem from different starting points. Some document planning and execution workflows. Others emphasize test generation, execution, and assertions rather than a named plan-implement-verify decomposition.
The Magnitude testing framework, an open-source AI browser automation tool, implements a two-agent decomposition for testing. Per the founder, one agent handles planning and adapting test cases while another executes them quickly and consistently. Magnitude's Navigate, Interact, Extract, and Verify stages map onto the same triad.
The same decomposition appears across the five systems listed here.
| Tool/System | Plan Role | Implement Role | Verify/Heal Role |
|---|---|---|---|
| Playwright v1.56 | Planner | Generator | Healer |
| Magnitude | Planner agent | Executor agent | Planner re-invoked on failure |
| Teradata code assistant | Designer agent | Coder agent | Tester + Fixer agents |
| UiPath Agentic Testing | Plan | Run | Refine/Fix |
| TestQuality | Plan | Act | Verify |
UiPath independently describes its approach as using AI agents to "plan, run, and refine tests," while TestQuality describes "an Orchestration Layer that runs continuous Plan-Act-Verify reasoning loops to generate, execute, and validate test cases from plain-language requirements." Google Cloud's architecture documentation describes the coordinator pattern as a central agent decomposing a request into sub-tasks and dispatching them to specialized agents.
This convergence reframes how QA Leads should think about coverage strategy. The traditional test management tools model assumed humans authored tests at each layer. Agent-authored suites shift the unit of work from individual test scripts to coverage plans that a Coordinator decomposes and Implementors execute in parallel, with a Verifier gating each output.
For teams implementing continuous agent-authored testing, Augment Cosmos brings the same plan-implement-verify framing into the product workflow. E2E Testing, Deep Code Review, and PR Author experts run on that shared platform, with Context Engine providing repository context and a shared filesystem where tenant memory lets corrections from one test run compound across the team.
Build Your Coverage Strategy Around the Verifier Role First
Verifier-first coverage strategy makes intended behavior the admission gate for agent-authored tests. Agentic test authoring must demonstrate that generated tests validate the intended behavior rather than merely mirror the current code. CIV addresses that tension by giving verification its own agent and opposing pressure.
Related agentic-testing patterns follow the same shape: Playwright separates planning and generation while performing live verification and healing; Magnitude separates planning from execution; and academic frameworks describe plan-to-generation or generate-verify-revise flows.
Start by separating coverage planning from generation and adding a dedicated correctness check before tests enter your suite. The verifier should use requirements as its reference point.
Teams scaling verifier-first testing from one-off generation into continuous workflows with Augment Cosmos, let agent experts handle prioritization support, spec preparation, testing, review, and implementation while humans set goals and review checkpoints. Give verifier agents the requirements, related files, and dependency graph they need before generated tests enter production suites.
Frequently Asked Questions About CIV Test Authoring
Use the FAQ as a five-part implementation checklist.
- Define Coordinator, Implementor, and Verifier responsibilities before generation starts.
- Treat Playwright's Healer as post-execution repair and use a separate gate for pre-execution correctness verification.
- Separate the oracle review from the test generation to avoid self-certification.
- Anchor expected behavior in requirements instead of the current implementation logic.
- Apply multi-agent testing when orchestration costs buy independent planning or verification value.
Related Guides
Written by
Molisha Shah
Molisha is an early GTM and Customer Champion at Augment Code, where she focuses on helping developers understand and adopt modern AI coding practices. She writes about clean code principles, agentic development environments, and how teams are restructuring their workflows around AI agents. She holds a degree in Business and Cognitive Science from UC Berkeley.