AI governance tools built on rule engines assume a human authored each change; agent-written code needs a platform layer that records authoring context plus session-level actions and approvals, because an artifact gate evaluates only what arrives at the boundary.
TL;DR
Artifact gates evaluate only the finished code that reaches the decision point, which is where an agent-authored change looks identical to a hand-written one. The record that separates them sits in the system that ran the agent, so the shortlist question is which candidates enforce at that layer and which only report.
Say an agent opens a pull request against the payments service at 02:14. The Rego policy in the admission pipeline evaluates the diff: no disallowed base images or unapproved dependencies, and all required labels are present. The gate passes it. The gate never received the agent’s cross-repository context, four retries of a failing migration script, or its rewrite of the assertion in the test the requester asked it to satisfy. The artifact was clean. The gate had no authoring trail to evaluate.
This guide is for platform, security, compliance, and engineering leaders shortlisting AI governance tools for code that agents write, and it builds on the wider problem of AI-generated code risk. It does not cover secure SDLC controls across the lifecycle, ownership models, attestation design, or scanner selection.
What Governing Agent-Written Code Requires
A control enforces policy by blocking an action before execution or reversing it afterward. Reporting observes through dashboards, advisory comments, and alerts that fire after the merge. Vendors market both categories under the same governance label, so every criterion here turns on that distinction.
Rule engines assume a reviewer can question the human who authored a change. An agent compresses context gathering, tool calls, retries, test edits, and other authoring decisions into an artifact that looks hand-written. The system running the agent holds the full record. Evaluators should weigh candidates on enforcement at that layer and whether the record would survive an audit, which is where a tool shortlist meets an existing code governance framework. This guide evaluates Cosmos, Augment Code's unified cloud agents platform, as one candidate.
Why Artifact-Gate Governance Breaks Down
A policy engine evaluates the structured document that arrives at a decision point. Agent-written code complicates that model because relevant facts can live in the authoring process. A paper accepted to ACL 2026 surveyed seven major coding assistants and found that all of them gather context through automatic heuristics, often without developer awareness, and that none provide a way to view, limit, or log the gathered context. Its attack succeeded on 73.20% of attempts against eight models including GPT 4.1 and Claude 3.5 Sonnet v2, measuring semantics-preserving code transformations that induced vulnerable output on benchmark tasks rather than any rate of real-world compromise.
A preprint on self-admitted GenAI usage, last revised in April 2026, examined more than 200,000 GitHub repositories and found 1,292 self-admissions across 156 repositories. The same authors examined code churn in 151 of those repositories and found no general increase, so the finding establishes a disclosure gap rather than a quality effect.
The Nx supply-chain attack of August 26, 2025, exploited a GitHub Actions pull_request_target workflow to exfiltrate an npm publishing token. Malicious packages stayed live for roughly four hours. The S1ngularity postmortem attributes the exposure to three conditions in Nx's own repository: a PR title validation workflow using pull_request_target, unsanitized echoing of the PR title, and workflow permissions that granted a read/write token. The Nx postmortem does not establish a causal link between missing authoring metadata and the attack.
Prerequisites
Three things need to be in place before the first vendor demonstration. The team needs permission to create a deliberately violating change in a non-production environment, access to merge and deployment logs, and named owners who can say whether a control blocked an action or only reported it. Where a candidate is already deployed, including Cosmos, its purchased plan and its current training and retention policy belong in the file before testing starts.
Use one representative production-affecting workflow. A synthetic demo that never touches the organization’s real approval chain cannot establish whether a requester can bypass the control.
Enforcement vs. Reporting: The First Criterion
Evaluators should use this criterion to build the shortlist: include only tools that can prevent an action before execution or reverse it afterward. Treat everything else as observability layered on top. Enforcement takes one of two forms, and a demonstration against a failing policy makes either one observable.
- Prevent or gate: The tool blocks a merge or deployment, or holds it behind mandatory approval, and the requester cannot dismiss the control.
- Roll back: The tool reverses an applied change and records the actor responsible for the reversal.
Established policy-as-code tools encode the distinction with precision. HashiCorp documents Sentinel’s three enforcement levels separately from policy logic. An advisory policy can fail with only a warning. A mandatory policy must pass: soft mandatory permits a specified override, while hard mandatory requires someone to remove the policy. HCP Terraform policy sets can also permit privileged users to override a failed hard-mandatory check. That vocabulary forces a named level for each advertised control.
Article 14(4) of the EU AI Act requires oversight capabilities that let a person disregard, override, or reverse output and interrupt the system through a stop button or similar procedure. These duties apply only when the AI system qualifies as high-risk under the Act. Standard software development use does not by itself match the listed high-risk categories, and the Commission’s implementation guidance states that the AI Act does not regulate source code when a supplier provides it as source code alone. Employment-related use or use as a regulated product safety component can change the classification. For an in-scope high-risk system, a reporting-only control does not provide the intervention capability Article 14 describes.
Where Open Policy Agent and HashiCorp Sentinel Stop
Open Policy Agent’s documentation defines it as “an open source, general-purpose policy engine that unifies policy enforcement across the stack.” OPA decouples policy decision-making from enforcement. Integrating software queries OPA and supplies arbitrary structured data, such as JSON, as input. Common documented examples include Kubernetes AdmissionReview objects, Terraform plan JSON, HTTP API requests, and infrastructure-as-code files.
HashiCorp’s Sentinel documentation describes an embeddable policy-as-code framework available as an enterprise-only feature of HashiCorp Consul, Nomad, Terraform, and Vault. After Terraform successfully creates a plan in HCP Terraform, Sentinel checks whether the plan complies with policy before Terraform can apply it.
| Dimension | Open Policy Agent | HashiCorp Sentinel |
|---|---|---|
| Input model | Arbitrary structured input supplied by integrating software | Product run context, including imports and external data |
| Common enforcement point | Admission or API authorization layer; CI/CD integration | Embedded HashiCorp product workflow |
| Terraform workflow | Evaluates Terraform plan JSON when platform engineers integrate it into the workflow | Runs after terraform plan in HCP Terraform |
| Enforcement owner | The calling software enforces OPA’s decision | The embedding HashiCorp product enforces the policy result |
| Authoring context | Available only when the integrating system supplies it | Available only through configured imports or product context |
| Best use | Cross-stack policy decisions over structured data | Policy enforcement inside supported HashiCorp products |
Both engines render decisions from the structured input that the integrating software presents at a decision point. An agent-authored Terraform plan can look identical to a hand-written one. Standard AdmissionReview objects and plan JSON omit model identity, prompt scope, files read, and retry history unless an integrating system adds those fields.
A Rego or Sentinel rule can evaluate authoring context when the calling system supplies it. Buyers should retain artifact gates and separately evaluate a layer that produces authoring-context records as policy inputs.
Core Capabilities to Verify Before Shortlisting AI Governance Tools
Buyers should require every candidate to clear a minimum capability bar before deeper evaluation. Evaluators should remove any tool missing an item below from the shortlist.
- AI registry: A live inventory of models and agents, including their tools. NIST AI 600-1 recommends recording data provenance, known issues, oversight roles, and foundation model versions.
- Risk tiering: Documented criteria separating acceptable from non-acceptable risks, which ISO/IEC 42001 requires without prescribing tier labels.
- Audit trails: Automatic event recording over the system’s lifetime for qualifying high-risk systems, as Article 12 of the EU AI Act requires, with a six-month minimum retention floor that falls on providers under Article 19 and on deployers under Article 26(6).
- Intake workflows: A registry entry that triggers review before use, so onboarding a new agent becomes a governed event.
- Production behavior monitoring: Monitoring system functionality and behavior in production under AI RMF 1.0 MEASURE 2.4, with harmful bias among the risks that the Generative AI Profile names.
- Regulatory mapping: Explicit mapping to applicable EU AI Act requirements, NIST AI RMF guidance, ISO/IEC 42001 controls, and DORA obligations where relevant.
A vendor that cannot demonstrate every capability against the purchased tier should not proceed to workflow testing.
A presumption of conformity attaches only to harmonized standards referenced in the Official Journal, and the Commission's standardization FAQ still describes those standards in the future tense, so certification against any standard available today does not carry it. The same FAQ states that ISO/IEC 42001's goals and definitions do not align with the quality management system the Act requires, which is why the Commission has requested a new standard aimed specifically at regulatory compliance. Buyers should require vendors to demonstrate each claimed mapping and identify the evidence the tool creates.
Shadow AI Discovery Feeding Enforcement
Gartner’s 2026 AI Usage Control category covers tools that apply organizational security policies to third-party AI use, including discovery and shadow AI cataloging. A Gartner survey of 302 cybersecurity leaders across North America, EMEA, and Asia/Pacific, conducted from March through May 2025, found that 69% of organizations suspected or had evidence that employees used prohibited public GenAI.
Discovery that populates a dashboard is reporting. The discovery service should open an intake case for each newly found agent, and the governance owner should assign its risk tier. The credentialing system should keep the agent from obtaining production credentials until both steps finish. A Cosmos demonstration should show that connection before evaluators classify the platform as enforcement.
Human-in-the-Loop Gates: Mandatory vs. Advisory
Gates must block stages that touch production or modify AI systems. BaFin guidance dated January 23, 2026, frames its expectations against the DORA regulatory technical standards on ICT risk management. The guidance treats any change to an AI system as generally subject to strict change management, including independent review, documented testing, and specified fall-back procedures.
The AI Omnibus, which entered into force on July 27, 2026, moved the Annex III high-risk obligations under Articles 12, 14, and 26 from August 2, 2026 to December 2, 2027. Article 26(2) requires deployers of qualifying high-risk systems to assign oversight to people with the necessary competence and authority, supported by appropriate training. A gate qualifies only when policy sets its mandatory status and the requesting user cannot unset it.
Reading the Authoring Context Alongside the Artifact
The platform-layer criterion follows from where the evidence lives. The joint Five Eyes advisory on agentic AI adoption, published on May 1, 2026, names accountability risk among five risk categories. Opacity makes it difficult to trace decisions, audit actions, or assign responsibility once actions run autonomously and at scale. The remedy sits upstream of an artifact gate: the system running the agent must record its actions while policy can still intervene, which is a question of agent observability and not scanner coverage.
Cosmos is Augment Code's unified cloud agents platform, generally available on paid plans. Environments govern agent scope, while Experts define behavior. Sessions turn one-off prompts into workflows that carry an audit trail. Evaluators must still verify whether the purchased Cosmos tier makes a given approval mandatory and prevents the requester from bypassing it.
A scanner that sees only the finished artifact cannot flag prompt injection vulnerabilities whose evidence it never receives. When testing any platform-layer candidate, including Cosmos, evaluators should trace a completed agent task end to end from its audit trail and inspect the actions, the approvals, the governing policy, and the final outcome.
Vendor and Third-Party AI Risk: Evidence to Require
Due diligence on externally sourced models, APIs, agent frameworks, and coding assistants runs on documents. A complete vendor file pairs the audit report, the training policy, a current-report verification, and contract terms scoped to the purchased tier.
- The audit report with auditor and date: Coalfire issued Augment Code’s ISO/IEC 42001 certification in August 2025. Coalfire’s announcement states that the Stage 2 audit assessed risk management, system impact assessments, development workflows, data management, and oversight of third-party AI components. Augment Code also holds SOC 2 Type II.
- Training policy by plan tier, with the policy date: GitHub's Copilot policy docs state that starting April 24, 2026, GitHub may use interactions from the Copilot Free, Pro, Pro+, and Max plans to train and improve AI models unless the subscriber opts out in personal settings. GitHub does not use Business and Enterprise customer data for model training.
- Verify the current audit report: Cognition's security page states that Cognition obtained SOC 2 Type II in March 2024, while the data processing agreement it published on March 9, 2026, states that Cognition maintains SOC 2 Type I. Buyers should verify the current report through the vendor’s trust portal.
- Contract terms matching the purchased tier: The contract should identify retention periods, training restrictions, encryption key control, and the notice period for policy changes.
An auditor’s name, report date, policy version, and contract language make vendor commitments testable.
How to Apply This: A Buyer’s Evaluation Sequence
A team can apply the sequence below this week.
- Inventory current agent tools. List every AI coding tool and agent in use, sanctioned or not, with the plan tier and policy date for each. In a Gartner Peer Community survey of 123 respondents, 67% lacked a formal GenAI governance policy. Write the inventory first because a policy without a current tool list governs an incomplete environment.
- Test whether each control blocks or observes. Author a deliberately violating change, using an agent where possible, and attempt to merge and deploy it. Classify each control by what stopped the change. A tool that commented while the merge proceeded belongs in the reporting column.
- Map framework coverage in writing. For applicable EU AI Act duties, NIST AI RMF guidance, ISO/IEC 42001 controls, and sector-specific rules, require each vendor to state which controls produce evidence automatically and which require manual assembly. Get the answer as a document.
- Require audit-chain and CMEK evidence. Ask for the full chain from agent action to merged change, which the team can export to the SIEM. Confirm that customer-managed encryption keys are available on the purchased tier, and require an audit report that names the auditor and date.
Where a shortlist needs an authoring-context layer, run the same sequence against Cosmos. Ask the demonstration to show a mandatory human approval on a production-affecting action, and watch the agent halt against it. Then send the violating change through a Cosmos-managed agent and set its captured workflow beside the artifact gate's log. The distance between those two logs is what separates a platform-layer record from an artifact-layer one, and one demonstration makes it visible.
Where Governance Programs Look Complete and Are Not
Evaluators can use recurring failures to determine whether teams wired governance controls into real workflows.
- Treating discovery as reporting. Compliance teams often assemble an AI inventory for audit season that no workflow consumes, while procurement teams buy discovery tooling for visibility without connecting it to intake. The correction is a registry entry that opens a risk-tiering case and withholds production credentials until the tier is assigned.
- Advisory-only gates on high-risk stages. In July 2025, Replit's agent deleted a production database during an explicit code and action freeze. The affected user's own account of the incident puts the loss at 1,206 executive records and roughly 1,200 companies, alongside 4,000 fabricated records. The freeze instructed the agent without enforcing a boundary. Replit added default development and production database separation after the incident. Production-affecting stages need hard-mandatory gates, and instructions to an agent never count as controls.
- Failing to verify tier-specific no-training policies. A blanket "we never train on your code" answer fails to identify the purchased tier or future policy changes. Verify the exact tier, record the policy version date, and move the verified commitment into the contract so it stays enforceable after procurement.
Platform and procurement teams should connect governance controls to credentials, to mandatory approvals, and to contractual terms.
What to Do Next
Choose one production-affecting repository and route a single agent-authored change through the existing controls this week. Record four fields before reviewing the result: model identity, prompt scope, files read, and retry count. Mark every field the current logs cannot answer, then use those gaps as acceptance criteria in the first vendor demonstration.
Frequently Asked Questions
Related Reading
Written by
Molisha Shah
Molisha is an early GTM and Customer Champion at Augment Code, where she focuses on helping developers understand and adopt modern AI coding practices. She writes about clean code principles, agentic development environments, and how teams are restructuring their workflows around AI agents. She holds a degree in Business and Cognitive Science from UC Berkeley.