Treat model and agent output as untrusted at every sink it reaches

Evidence grade

Standards consensus (7)

7 standards and frameworks · 1 graded incident.

TwinEthos recommendation, not law. Where binding law applies, the law governs. No binding law in the corpus requires this control yet. 7 standards and frameworks recommend it (MITRE ATLAS (2026.09), NIST AI 600-1, NIST AML Taxonomy (AI 100-2e2025) — Agentic, NIST SP 800-218A, OWASP Agentic Top 10 (2026), OWASP LLM 2026, OWASP LLM Top 10 (2026)). 1 graded incident cited. Context: binding law on related controls in the family “Generated output is acted on without validation or leakage screening” is in force in 2 jurisdictions (AU, IN).

Standards and frameworks

• Model output should be validated and encoded for the interpreter it reaches (OWASP LLM Top 10 (2026); OWASP Top 10 for LLM Applications 2026 — LLM10:2026 Improper Output Handling: Prevention and Mitigation Strategies, strategies 1, 3, 4, 5 and 9; same control)
• Inputs and outputs of agent tools and data sources should be validated outside the agent (MITRE ATLAS AML.M0033) (MITRE ATLAS (2026.09); MITRE ATLAS 2026.09, AML.M0033 Input and Output Validation for AI Agent Components (description); same control)
• Model output that downstream code uses should be validated, and rendered output should not leak data to attacker URLs (NIST AI 100-2e2025) (NIST AML Taxonomy (AI 100-2e2025) — Agentic; NIST AI 100-2e2025, Sec. 3.1.1, item 3 (output handling); same control)
• AI model inputs and outputs should be validated and encoded so they cannot execute unauthorized code (NIST SP 800-218A PW.5.1) (NIST SP 800-218A; NIST SP 800-218A, PW.5 and PW.5.1 (task); same control)
• Agent-generated code and commands should never reach an interpreter unvalidated (OWASP Agentic Top 10 (2026); OWASP Top 10 for Agentic Applications for 2026 — ASI05: Unexpected Code Execution (RCE): Prevention and Mitigation Guidelines, guidelines 1 and 3; same control)

Published AI security standards mapping to this control

• OWASP LLM 2026 LLM01: Prompt Injection · crosswalk status: covered
• OWASP LLM 2026 LLM07: Misinformation · crosswalk status: covered
• OWASP LLM 2026 LLM10: Improper Output Handling · crosswalk status: covered
• OWASP ASI 2026 ASI05: Unexpected Code Execution (RCE) · crosswalk status: covered
• MITRE ATLAS AML.M0032: Segmentation of AI Agent Components · crosswalk status: covered
• MITRE ATLAS AML.M0033: Input and Output Validation for AI Agent Components · crosswalk status: covered
• NIST AI 600-1 2.9: Information Security · crosswalk status: covered
• NIST AI 100-2 3.1.1 output handling: Model output used dynamically in web pages or executed commands without human supervision · crosswalk status: covered
• NIST AI 100-2 3.2.3: Supply-chain mitigations: verify downloads by hash, scan model artifacts, treat models as untrusted components · crosswalk status: covered
• NIST AI 100-2 3.4.3: Exfiltration through Markdown image rendering and attacker URLs · crosswalk status: covered
• NIST SP 800-218A PW.5.1: Handle, validate, and encode AI inputs and outputs · crosswalk status: covered

Item ids and titles from the published standards; the mapping is TwinEthos's (standards crosswalk, docs/COVERAGE.md Part 4). Cited by id, never quoted.

Family “Generated output is acted on without validation or leakage screening”: binding law on related controls is in force in Australia (AU), India (IN). Context only: it does not change this guardrail's grade.

Graded incidents

• Slack AI indirect prompt injection (researcher disclosure) (2024-08; confirmed) PromptArmor (original researcher disclosure) · evidence grade: primary

The guard to add

{messages.map(m => (
  <div key={m.id}
    dangerouslySetInnerHTML={{ __html: marked.parse(m.content) }} />
))}

import ReactMarkdown from 'react-markdown';   // escapes raw HTML by default; no rehype-raw
{messages.map(m => (
  <ReactMarkdown key={m.id} disallowedElements={['img']} unwrapDisallowed>
    {m.content}
  </ReactMarkdown>   // no auto-loaded images: model text cannot beacon data out
))}

Why

Model output is shaped by whoever wrote the prompt, the retrieved document or the web page the model read, so code that trusts it hands each of them a way into the systems downstream: a rendered image tag that sends data to an attacker's server, a query or shell command built from model text, generated code run in the application's own process. Output handling is the item OWASP ranks in its LLM list as improper output handling, and NIST, MITRE ATLAS and OWASP's agentic list all recommend validating, encoding and sandboxing what models produce. Researchers have shown an instruction planted in content an assistant read making it leak private data through a crafted link.

Class: agent security · set: ai security · maturity: reviewed · confidence: high · id guardrail.sec-model-output-handled-as-untrusted