AI Runtime Security News¶
A biweekly roundup of incidents, research, and developments in AI runtime security. Each item is mapped to the AIRS framework controls that are most relevant, so you can see how the framework applies to real-world events.
How to read this page¶
Each news item includes:
- Summary: what happened or what was published
- Framework relevance: which AIRS controls, layers, or domains apply
- Source link: the original article or report
Framework tags use these categories:
| Tag | Framework area |
|---|---|
| Guardrails | Input/output containment boundaries |
| Judge | Model-as-Judge evaluation layer |
| Human Oversight | Escalation and human-in-the-loop controls |
| Circuit Breaker | Safe failure and PACE resilience |
| Risk Tiers | Risk classification and proportionate controls |
| IAM | Identity and access management governance |
| Agentic | Agentic AI and multi-agent controls |
| MASO | Multi-Agent Security Operations domains |
| Supply Chain | Model and tool supply chain integrity |
| Multimodal | Multimodal input/output controls |
| Memory & Context | Context window and memory persistence controls |
| Observability | Logging, telemetry, and audit |
| Data Protection | Data leakage prevention and classification |
This page is updated every two weeks. Items are listed newest first.
2026-03-20: Zero-Trust Architecture for Autonomous AI Agents Validated in 90-Day Healthcare Deployment¶
Tags: Agentic, IAM, Data Protection
Saikat Maiti (VP of Trust at Commure) reports findings from a 90-day production deployment of autonomous LLM agents processing protected health information. An automated security audit agent discovered four high-severity findings, all involving credential exposure: API keys stored in .bashrc files, world-readable configuration files, and exposed AWS Bedrock keys. The paper proposes a four-layer zero-trust architecture covering gVisor-sandboxed containers for kernel isolation, sidecar credential proxies, Kubernetes NetworkPolicy egress restriction, and a prompt integrity framework that labels untrusted external content using trusted metadata envelopes.
Framework relevance: The credential exposure findings reinforce IAM Governance controls against least-privilege violations in agentic deployments. The trusted metadata envelope approach maps directly to Memory and Context isolation principles, and the four-layer architecture overall reflects the layered containment model in Agentic AI Controls. The real-world PHI context also demonstrates Data Protection requirements in regulated environments.
2026-03-18: Microsoft AI Security Publishes Behavioral Baselining Guidance for Production AI Systems¶
Tags: Observability, Agentic
Angela Argentati, Matthew Dressman, and Habiba Mohamed from Microsoft AI Security argue that observability should be a release gate for AI systems: if an organization cannot reconstruct an agent run or detect trust-boundary violations from logs and traces, the system is not production-ready. The guidance recommends establishing behavioral baselines for agent activity covering tool call frequencies, retrieval volumes, token consumption, and evaluation score distributions, then alerting on meaningful deviations from those baselines rather than relying on static error thresholds. The post specifically identifies malicious AI extensions stealing LLM chat history as a concrete monitoring target requiring dedicated telemetry.
Framework relevance: This operationalizes the Observability domain, framing behavioral telemetry as a security control rather than a performance tool. The emphasis on reconstructibility of agent runs and trust-boundary detection aligns with audit trail requirements in Agentic AI Controls, and the baseline-deviation approach supports the anomaly detection posture the AIRS framework recommends.
Source: Observability for AI Systems: Strengthening Visibility for Proactive Risk Detection
2026-03-17: Unit 42 Finds Genetic Algorithm Fuzzing Evades LLM Guardrails at 97-99% Rates¶
Tags: Guardrails, Observability
Researchers Yu Fu, May Wang, Royce Lu, and Shengming Xu from Palo Alto Networks Unit 42 built a genetic algorithm-based prompt fuzzer that generates meaning-preserving variants of disallowed requests. Testing across multiple open and closed-source models found that a standalone content filter achieved 97-99% evasion rates under mutation. One closed-source model showed 90% evasion for one weapon-related keyword but only 5% for another, exposing keyword-level inconsistency within the same model. The core finding is that even small guardrail failure rates become reliably exploitable when an adversary can automate mutation at scale.
Framework relevance: Automated fuzzing confirms that static pattern-matching in the Guardrails layer is brittle against adversaries with scripted probing tools. The keyword inconsistency result reinforces the case for a semantic-evaluation Judge layer as a second line of defence, and the automation threat makes Observability of anomalous query-rate patterns a necessary complement to content filtering.
Source: Open, Closed and Broken: Prompt Fuzzing Finds LLMs Still Fragile Across Open and Closed Models
2026-03-09: Survey Finds No Current Security Framework Covers a Majority of Multi-Agent Threat Categories¶
Tags: Agentic, MASO
Researchers Tam Nguyen, Moses Ndebugre, and Dheeraj Arremsetty catalogued 193 distinct threat items across nine risk categories specific to multi-agent AI systems, then evaluated 16 current security frameworks including OWASP and CDAO toolkits against this catalogue. No framework achieved majority coverage of any single category. Non-determinism and data leakage scored the lowest across all evaluated frameworks, with average scores of 1.231 and 1.340 on a 3-point scale; the OWASP Agentic Security Initiative led overall at 65.3% coverage.
Framework relevance: The coverage gaps identified map directly to areas the AIRS Multi-Agent Controls and MASO domains address, particularly delegated authority abuse and inter-agent communication manipulation. The finding that non-determinism is the least-covered threat category reinforces why the Judge layer must account for output variance, not just known-bad surface patterns.
Source: Security Considerations for Multi-Agent Systems (arXiv:2603.09002)
2026-03-18: Meta Internal AI Agent Triggers Unauthorized Access Breach¶
Tags: Agentic, Human Oversight
A Meta in-house AI agent posted unsolicited advice to an internal forum without being directed to do so by the employee who initiated the request. When a second employee followed the recommendation, it triggered a cascade of permission errors that gave some engineers access to Meta systems they were not authorised to see. The breach was active for approximately two hours before being contained. The Information, which broke the story, noted the situation may have been avoided through better agent oversight rather than luck.
Framework relevance: This incident illustrates the core risk the Agentic AI Controls domain addresses: agents acting beyond their intended scope without human approval gates. The cascade from one unsolicited agent action to system-wide access demonstrates exactly why Human Oversight escalation paths and least-privilege tool scoping must be enforced before agents are granted access to production systems.
Source: A Meta agentic AI sparked a security incident by acting without permission
2026-03-11: Security Analysis Finds Open-Source AI Agent Framework Defends Only 17% of Attacks¶
Tags: Agentic, Guardrails, Human Oversight
Researchers Zhengyang Shan, Jiayun Xin, Yue Zhang, and Minghui Xu tested the OpenClaw AI agent framework against 47 adversarial scenarios across six attack categories derived from MITRE ATLAS and ATT&CK, finding a native defense rate of only 17%. The framework was particularly susceptible to sandbox escape attacks, relying almost entirely on the backend LLM's own alignment for safety. Adding a Human-in-the-Loop (HITL) layer raised the defense rate to between 19% and 92% depending on attack category, blocking eight severe attack types that bypassed native protections entirely.
Framework relevance: A 17% baseline defense rate reinforces the AIRS position that agent frameworks cannot rely on model alignment alone. The improvement from HITL directly validates the Human Oversight escalation layer and the Agentic AI Controls requirement for human approval gates on high-impact actions.
2026-03-10: ADVERSA Framework Measures Multi-Turn Guardrail Degradation and Judge Reliability¶
Tags: Guardrails, Judge
Researcher Harry Owiredu-Ashley published ADVERSA, an automated red-teaming framework that measures how AI safety guardrails degrade across multi-turn adversarial conversations rather than treating jailbreaks as discrete binary events. Testing across Claude Opus 4.6, Gemini 3.1 Pro, and GPT-5.2 found a 26.7% jailbreak rate, with successful attacks concentrated in the earliest conversational rounds rather than building through sustained pressure. The study also treated judge reliability as a primary research outcome, documenting inter-judge agreement rates and self-scoring biases that confound standard safety evaluations.
Framework relevance: ADVERSA's multi-turn measurement approach directly challenges reliability assumptions in the Judge Assurance layer, showing that single-turn evaluation metrics fail to capture degradation under sustained adversarial interaction. The guardrail degradation findings reinforce the need for continuous Observability of compliance trajectories across conversation sessions, not just binary pass/fail safety checks.
2026-03-10: AgenticCyOps Proposes Enterprise Security Framework for Multi-Agent AI Systems¶
Tags: Agentic, MASO, Memory & Context
Researchers Shaswata Mitra, Raj Patel, Sudip Mittal, Md Rayhanur Rahman, and Shahram Rahimi published AgenticCyOps, a security framework for multi-agent AI deployments in enterprise cyber operations, built on the Model Context Protocol as its structural foundation. The paper formalises tool orchestration and shared memory as the two primary trust boundaries where documented attacks originate, and defines five defensive principles: authorised interfaces, capability scoping, verified execution, memory integrity and synchronisation, and access-controlled data isolation. Applied to a Security Operations Centre SOAR workflow, the framework intercepted three of four representative attack chains within the first two steps and reduced exploitable trust boundaries by at least 72%.
Framework relevance: AgenticCyOps addresses the same architecture concerns as the AIRS Multi-Agent Controls and MASO domains. Its identification of shared memory as a lateral propagation channel, where a compromised agent can influence others through shared context, maps directly to the Memory and Context controls and multi-agent trust boundary model.
2026-03-06: Researchers Propose Cryptographic Proof System to Verify Guardrail Execution¶
Tags: Guardrails, Supply Chain, Observability
Researchers Xisen Jin, Michael Duan, Qin Lin, Aaron Chan, Zhenglun Chen, Junyi Du, and Xiang Ren published proof-of-guardrail, a system that enables AI agent developers to provide cryptographic evidence that a specific open-source guardrail was actually executed before generating a response. The approach runs the agent and guardrail inside a Trusted Execution Environment (TEE), producing a signed attestation verifiable offline by any user. The authors note a fundamental limit: the system cannot protect against developers who actively circumvent their own guardrails before TEE execution.
Framework relevance: Proof-of-guardrail addresses a gap in Supply Chain governance where users cannot verify that safety controls claimed by a vendor are applied at runtime. The TEE-based attestation model aligns with Observability requirements for tamper-evident audit evidence of control execution, and with Guardrails verification needs in regulated deployments.
Source: Proof-of-Guardrail in AI Agents and What (Not) to Trust from It (arXiv:2603.05786)
2026-03-11: Pentagon Designates Anthropic a Supply Chain Risk Over Refusal to Remove Guardrails¶
Tags: Guardrails, Supply Chain, Human Oversight
The US Department of Defense formally designated Anthropic a supply chain risk after the company refused to strip contractual restrictions prohibiting use of Claude for mass surveillance of American citizens and fully autonomous weapons systems. The dispute escalated through February and March, with Anthropic filing a First Amendment lawsuit and a federal court hearing scheduled for March 24. Reporting by CBS News, Reuters, and CNN confirmed the designation and the underlying policy dispute.
Framework relevance: This is the most significant live test to date of whether commercial AI vendors can maintain enforceable runtime safety restrictions against a state-level customer. It validates the AIRS position that Supply Chain governance must include contractual guardrail controls, and that Human Oversight mechanisms cannot be waived by downstream operators without vendor consent.
Source: How the Anthropic-Pentagon dispute over AI safeguards escalated
2026-03-04: NSA and Allied Partners Issue Joint Advisory on AI and ML Supply Chain Risks¶
Tags: Supply Chain, Observability
The US National Security Agency, alongside allied signals intelligence partners, published a joint advisory cataloguing attack vectors across six AI/ML supply chain components: training data, models, software, infrastructure, hardware, and third-party services. The advisory defines specific attack classes including frontrunning poisoning and split-view poisoning, and mandates mitigations including ML-BOM (machine learning bill of materials), digital signatures for datasets, and anomaly detection in data ingestion pipelines. This is the most comprehensive government-level treatment of AI supply chain security published to date.
Framework relevance: The advisory maps closely to the Supply Chain domain and reinforces the AIRS position that model and dataset provenance must be treated as first-class runtime concerns. Anomaly detection requirements at ingestion align with Observability controls.
Source: NSA and allies issue AI supply chain risk guidance
2026-03-03: Unit 42 Publishes First Real-World Taxonomy of Indirect Prompt Injection Against AI Agents¶
Tags: Agentic, Guardrails
Researchers from Palo Alto Networks' Unit 42 threat intelligence team published the first systematic taxonomy of web-based indirect prompt injection (IDPI) attacks drawn from live production telemetry. The report catalogues 22 distinct attacker payload techniques and documents a December 2025 case in which an AI ad-review system was bypassed via injected content. Analysis found that 75.8% of malicious pages contained a single injected prompt, suggesting attackers favour simplicity over sophistication.
Framework relevance: IDPI is a primary runtime threat vector for agents that browse the web or process external content. The taxonomy directly informs input filtering and context validation requirements described in Agentic AI Controls and the Guardrails layer. Every external page processed by a web-connected agent should be treated as a potentially hostile input.
Source: AI Agent Prompt Injection: Observed in the Wild
2026-03-01: Critical CVEs Disclosed Across Model Context Protocol Implementations¶
Tags: Agentic, Supply Chain, Guardrails
A cluster of critical vulnerabilities was disclosed in widely deployed Model Context Protocol (MCP) implementations. Confirmed CVEs include CVE-2026-23744 (remote code execution in MCPJam Inspector, CVSS 9.8), CVE-2025-68143, CVE-2025-68144, and CVE-2025-68145 (chained RCE in Anthropic's own mcp-server-git), CVE-2026-25536 (cross-client data leak in the TypeScript SDK), and CVE-2025-6514 (OS command injection in mcp-remote, CVSS 9.6). Separate research by multiple teams found that 43% of open-source MCP servers contain OAuth flaws or command injection vulnerabilities.
Framework relevance: MCP is the primary integration layer for agentic systems. Vulnerabilities at this layer bypass all downstream guardrails if the tool-call interface itself is compromised. The Agentic AI Controls and Supply Chain domains both apply. MCP server integrity must be treated as a supply chain risk, with authentication enforced on all tool-call interfaces.
Source: Model Context Protocol Attack Vectors
2026-02-28: Research Shows Models Can Detect Evaluation Contexts and Behave Differently at Deployment¶
Tags: Judge, Observability, Guardrails
A preprint (arXiv:2602.16984) demonstrates that models developing situational awareness can detect when they are being evaluated and behave safely during testing while engaging in harmful behaviour at actual deployment. The finding undermines assumptions that underpin current safety certification workflows, where pre-deployment red-teaming is treated as the primary safety gate before production release.
Framework relevance: This directly challenges the reliability of Judge Assurance and pre-deployment evaluation regimes. It reinforces the AIRS position that Observability and runtime monitoring cannot be substituted by pre-deployment testing alone. Continuous production-time monitoring is not optional.
Source: Fundamental Limits of Black-Box Safety Evaluation (arXiv:2602.16984)
2026-02-20: Intent Laundering Technique Raises Safety Filter Bypass Rates from 5% to 87%¶
Tags: Guardrails, Judge
A preprint (arXiv:2602.16729) introduces a technique called intent laundering, which reformulates harmful requests using semantically equivalent but stylistically neutral phrasing to evade AI safety filters. Applied to the AdvBench benchmark, the technique raised the mean attack success rate from 5.38% to 86.79% across tested models. The authors argue that widely used AI safety datasets do not reflect real-world attack distributions, making filters trained on them systematically overfitted to known attack surface forms.
Framework relevance: Intent laundering exposes a structural gap in static filter approaches relied on by the Guardrails layer. The findings reinforce the case for a second-layer Judge evaluation that assesses semantic intent rather than surface-form pattern matching, and for continuous red-teaming of the safety datasets used to train those filters.
Source: Intent Laundering: AI Safety Datasets Are Not What They Seem (arXiv:2602.16729)