Tool output poisoning: how untrusted data hijacks AI agents
A malicious API, MCP or CLI result can redirect an agent. Secure tool output with schemas, provenance, isolation and independent action policy.
- AUTHOR
- Karol Rapacz / Penetration Tester (OSCP, PNPT)
- PUBLISHED
- 17 June 2026
- READING TIME
- 9 min read
- TOPIC
- AI Security
Tool output poisoning occurs when API, MCP, CLI or search output contains instructions influencing an agent’s next step. The model sees a response as an observation but may treat it as a higher-priority command.
Attack sources
Malicious text can live in filenames, issue descriptions, database records, stderr, tool descriptions or errors. An attacker only needs control of data retrieved by a tool.
Structure output
Tools should return a small typed JSON schema with provenance and trust classification. Remove HTML, active Markdown and unnecessary fields. Process large text through an isolated component without tools.
Do not pass raw stderr to a privileged model. Map errors to codes and safe descriptions.
Authorise the next step
Compare proposed actions with the original user goal, not the last observation. A validator checks tool, parameters, resource and recipient. New domains or scopes require approval.
Testing
Inject instruction overrides, forged reasoning, exfiltration links and malicious parameters into every response field. Test multi-step chains and downstream agents.
OWASP calls these patterns thought/observation injection and tool poisoning. For MCP environments, combine controls with MCP hardening.
Why tool output is untrusted input
APIs, MCP servers, search engines and document parsers return data partly controlled by external parties. Even valid JSON may contain text instructing the model to reveal a secret, change the goal or call another tool. Transport authentication confirms the source; it does not make content a trusted instruction.
The orchestrator should label field provenance and keep data separate from control instructions. A model may analyse the result, but the policy layer authorises the next action from identity, task goal, permitted tool and argument schema. Text returned by a tool cannot create new privileges.
Validation and data reduction
Validate response structure, types, lengths, encoding and accepted values. Reject unexpected fields, active HTML and unapproved URIs. If the agent only needs a status and identifier, do not expose the full response with comments and metadata. Smaller context reduces both injection surface and accidental disclosure.
Before a high-impact action, build a new trusted parameter object. Do not copy a command, path or destination directly from a textual observation. Require matching origin and scope, plus explicit confirmation when the data recipient changes.
Regression matrix
Place malicious content in a value, field name, error message, pagination, tool description and streaming response. Check whether it survives summarisation, memory storage and transfer to a second agent. Test conflicting tools and a valid response containing a new-domain link.
Store raw source in a protected evidence system, while sending a normalised representation to the model. An alert should identify the observation that preceded the denied action and the policy rule that stopped it.
Safe multi-agent composition
Do not pass a free-form transcript as a trusted command. A sending agent returns schema-valid results, task identifier, provenance and confidence. The receiving agent gets data, while its own policy reassesses every action.
Delegation does not transfer permissions. If a research agent can read the web and a deployment agent can access a repository, the first agent’s output cannot become the second agent’s command. A controlled boundary must reduce the data and require new authorisation.
Poisoning response
Stop dependent calls and mark the complete trace. Identify source server, tool version, fields shown to the model and all later actions. Invalidate cache and memory that may contain the result.
Add the payload to regression tests with semantic variants and alternative placements. Blocking one sentence is fragile. A durable fix prevents privilege flow regardless of wording.
Every tool owner should publish a data contract and a process for unexpected responses. Schema changes, new field types and expanded tool descriptions trigger security tests just like executor changes. This prevents a routine integration update from creating a new instruction channel.
Sources: OWASP Prompt Injection Prevention, OWASP MCP Security.
