AI/Agentic Threat Modeling: Securing Systems That Include Agents

AI and agentic threat modeling covers the model and every runtime surface around it, including agents, memory, tools, and orchestration layers. The approach accounts for tool calls, memory writes, and autonomous decisions that can alter what the system can reach and change.

Agent-authored development workflows expose a timing gap. Modules appear in seconds while manual threat modeling takes hours. During that pre-repository window, agents can change code, call tools, write memory, and update dependencies before a repository checkpoint. IBM's Cost of a Data Breach Report 2025 found 13% of organizations reported breaches of AI models or applications; 97% lacked proper AI access controls. Netskope's AI Risk and Readiness Report 2026 found AI tools at 73% of organizations while real-time governance enforcement reached only 7%.

This guide covers why STRIDE-era methods fail against agents, how MAESTRO, MITRE ATLAS, and NIST AI RMF divide the work, what changes when agents write code, and a workflow teams can run today.

[ Meet Cosmos ]

Run your software agents at scale

Cosmos gives your agents the context, tools, and feedback loops they need to get better with every workflow.

Try it out

Why Agentic Threat Modeling Breaks Traditional Frameworks

Classical threat modeling assumes a relatively stable system boundary. An autonomous agent's attack surface expands dynamically with every tool negotiated and every memory layer populated. CSA describes the gap: STRIDE lacks the necessary scope to address threats unique to AI, such as adversarial machine learning, data poisoning, and the dynamic, autonomous behaviors of AI agents. Four failure modes recur:

• Goal hijacking has no STRIDE category: The OWASP Agentic Top 10 captures it as ASI01 (Agent Goal Hijack). No spoofing occurs, no data is tampered with at rest, and no privilege is elevated; the agent uses legitimate capabilities against you.
• Component analysis misses multi-step attack paths: If you STRIDE each component, an EchoLeak-style attack looks compliant. If you STRIDE the attack path, it does not. EchoLeak (CVE-2025-32711, CVSS 9.3) was a zero-click prompt injection in Microsoft 365 Copilot that exfiltrated internal files without user interaction.
• Trust boundaries are dynamic: Instructions like "never share sensitive data" are probabilistic controls; MCP research found tool poisoning evades output-based safety checks in 95% of cases.
• Persistence through memory has no classical category: The ATFAA framework defines temporal persistence threats as a distinct domain absent from STRIDE entirely.

STRIDE still applies at the component level for conventional databases and network flows. For classical methods, see the guide to threat modeling frameworks covering STRIDE, PASTA, and DREAD. This article focuses on threats those frameworks did not cover.

MAESTRO: Threat Modeling for AI Agents Across Seven Layers

CSA's MAESTRO framework (Multi-Agent Environment, Security, Threat, Risk, and Outcome) decomposes any agentic system into seven layers. CSA published it on February 6, 2025.

CSA calls Layer 7 the place where traditional security intuition breaks down most dramatically, because risk emerges from the composition of agents, tools, memory, and external integrations across organizational and trust boundaries. The February 2026 follow-on adds Implementation and Continuous Monitoring. Its cross-layer principle: the most dangerous attack paths start at Layer 1 and cascade through to Layer 4 and beyond.

CSA has published working MAESTRO models for the OpenAI Responses API and Google's A2A protocol.

MITRE ATLAS: AI Threat Modeling with Adversary Intelligence

MITRE ATLAS focuses on how real adversaries attack AI systems. As of v5.4.0, it contains 16 tactics, 84 techniques, 56 sub-techniques, 32 mitigations, and 42 case studies. ATLAS complements ATT&CK by adding two AI-specific tactics: AI Model Access and AI Attack Staging.

Agentic coverage expanded in v5.4.0. The MITRE and Zenity Labs collaboration added fourteen new techniques focused on AI agent security, including AI Agent Context Poisoning (AML.T0080), Modify AI Agent Configuration (AML.T0081), RAG Credential Harvesting (AML.T0082), Exfiltration via AI Agent Tool Invocation (AML.T0086), Escape to Host, and Publish Poisoned AI Agent Tool.

The case studies ground these techniques in incidents such as poisoned Postmark MCP email exfiltration (AML.CS0053), Bing Chat indirect prompt injection (AML.CS0020), and LAMEHUG malware attributed to Russian state-backed APT28 (AML.CS0044). Two ATLAS properties make the framework usable in engineering workflows: roughly 70% of mitigations map to existing security controls, and ATLAS data ships in STIX 2.1 format for machine-readable ingestion into detection pipelines. OWASP LLM01 (Prompt Injection) maps to three distinct ATLAS techniques at two tactic stages, so treating prompt injection as a single technique misses adversarial staging.

LLM Threat Modeling: The Core Threat Taxonomy

Teams usually classify LLM threats with the OWASP Top 10 for LLM Applications v2.0 (2025), the OWASP Top 10 for Agentic Applications 2026, and ATLAS technique IDs.

Indirect prompt injection and memory poisoning deserve attention in agentic contexts. Microsoft MSRC's July 2025 guidance identifies the deterministic fix: these attacks rely on the LLM-based application operating with the same level of access permissions as the user, which can be deterministically mitigated using fine-grained permissions and access controls.

Memory poisoning plants instructions into an agent's memory that survive across sessions and execute days or weeks later, triggered by unrelated interactions. MINJA research reported over 95% injection success rates across LLM-based agents and datasets, with over 70% attack success rate on most datasets. The OWASP Agentic Skills Top 10 names the compounding condition as the Lethal Trifecta: when an agent has private data access, untrusted content exposure, and external communication ability.

AI Security Risk Assessment with NIST AI RMF

NIST AI 100-1, released January 26, 2023, is voluntary and non-sector-specific. Its four functions: GOVERN establishes policies, roles, and risk tolerance as a cross-cutting function; MAP sets context for go/no-go decisions; MEASURE applies evaluation methods including MEASURE 2.7 for security and resilience; and MANAGE allocates resources to mapped and measured risks, with MANAGE 2.4 requiring mechanisms to supersede, disengage, or deactivate AI systems demonstrating inconsistent outcomes. The Generative AI Profile (NIST AI 600-1, July 2024) makes the framework concrete for LLM systems with 12 risk categories and over 200 actions.

The RMF sits above MAESTRO and ATLAS. CSA documents the layering: MAESTRO threat categories feed MAP, ATLAS validates coverage during MEASURE, and MANAGE allocates controls. For agentic deployments that can call tools, write memory, or change code at runtime, RMF output needs controls that apply mapped policies during execution. Augment Cosmos, the unified cloud agents platform, applies NIST AI RMF MANAGE controls through team-defined human-in-the-loop approval points within agent workflows.

Threat Modeling When Agents Author the System

Agent-authored systems need their own threat model because the development process itself becomes an attack surface. Veracode's 2025 GenAI Code Security Report found 45% of code samples failed security tests across 100+ LLMs and 80 coding tasks, with Java at a 72% failure rate. An IEEE-ISTAS 2025 study found a 37.6% increase in critical vulnerabilities after five iterations of LLM-based refinement. Agents that self-review their own code may degrade security with each pass.

Agent-authored supply chain risk concentrates around hallucinated dependencies. A USENIX Security 2025 study found hallucinated package names averaged 5.2% for commercial models and 21.7% for open-source models, with 205,474 unique fabricated names. Attackers register those names (slopsquatting).

Context Engine inside Cosmos workflows supports pre-merge checks for agent-authored changes through code review with 59% F-score quality, processing codebases across 400,000+ files through semantic dependency graph analysis. Cosmos Environments define repository, tool, and credential access before agents run.

A Practical AI Threat Modeling Workflow

A working AI threat model applies the frameworks above in sequence. Start from Adam Shostack's Four Question Framework and expand each answer for agents:

1. Decompose the system with agent-specific nodes: The agent runtime, memory, retrieval system, tool registry, approval path, scheduler, workers, and audit trail must all appear as distinct nodes in the data flow diagram.
2. Inventory the stack against MAESTRO's seven layers: Models, memory and vector stores, prompts, planners, tools, connectors, and orchestration; each maps to a layer with its own threat catalog.
3. Define trust boundaries with agentic precision: Tool responses, RAG retrievals, and inter-agent messages all cross boundaries. The application cannot inherently trust client-to-LLM input or LLM-to-client output.
4. Enumerate threats layer by layer, then hunt cross-layer chains such as infrastructure-to-data-to-model compromise paths.
5. Map every threat to ATLAS techniques to validate coverage against documented adversary behavior, and apply STRIDE per component for conventional elements.
6. Score and prioritize using OWASP's Agentic Vulnerability Scoring System.
7. Plan mitigations across four defense layers: input security, model security, tool security, and monitoring with audit trails for every tool call.
8. Map results into NIST AI RMF functions and define human approval for production data access, financial transactions above a threshold, external communications, and sub-agent spawning.

After scoring and mapping, keep the model continuous. CSA's CI/CD guidance sets the bar: every agentic AI codebase should have a continuously updated threat model, and every PR should show its threat delta. OWASP pytm, Threagile, and STRIDE-GPT automate selected steps; the guide to automated threat modeling covers the full tooling picture. Cosmos Sessions capture every approved agent run as an auditable, replayable workflow. The Org Knowledge Layer keeps continuous threat-model updates tied to tenant-shared memory, private memory, and organizational conventions. Cosmos is generally available and included on all paid plans.

Build the Threat Model Before the Next Agent Ships

For the next agent deployment this sprint, turn the threat model into runtime policy before the agent runs. Start with MAESTRO decomposition to identify the layers your agent touches, map those layers to ATLAS techniques, score using OWASP's Agentic Vulnerability Scoring System, and translate scored risks into NIST AI RMF MANAGE controls with defined human approval points. Cosmos Experts reuse agent configurations across the software development lifecycle, defining how agents behave, what tools they use, and which events they act on.

Frequently Asked Questions About AI Threat Modeling

These are the questions security teams and engineering leads ask when applying threat modeling frameworks to systems that include AI agents.

Related Guides

• Prompt Injection Detection
• Agentic SDLC Security
• Agent Execution Sandboxes
• AI Code Vulnerability Audit
• Agentic Attack Patterns