Danila (Dayfing)
Back to writing
2,493 words10 min

Prompt Injection in AI Agents and MCP: Threat Model and Defense

Prompt injection is not a malformed request that one regular expression can remove. It is a trust-boundary failure in which data changes a language model’s behavior. In an AI agent, that behavior can become a tool call, message, file write, or request to another service. The Model Context Protocol (MCP) eases tool discovery and invocation, but gives untrusted text more routes into a model and privileged systems.

This article applies the OWASP GenAI LLM Top 10 2026 to agents that read external content and use MCP. It is an engineering threat model, not a security audit. No control guarantees safety. Assume that an injection will sometimes influence the model, then make the resulting action difficult, visible, reversible, and limited.

Why the model is not a security boundary

An LLM receives system instructions, user requests, retrieved documents, tool descriptions, tool results, conversation history, and memory as tokens in one context. Delimiters and labels can communicate a trust policy, but they do not create an architectural barrier. The model can misunderstand a label, follow a plausible instruction in a document, or combine several harmless-looking clues into a dangerous plan.

OWASP LLM01:2026 describes prompt injection as behavior changed by direct input, retrieved content, tool output, images, audio, video, intermediate context, or persistent memory. The entry distinguishes the input-side failure from sensitive information disclosure, excessive agency, and improper output handling. Those categories often appear in one attack chain. An injection is the initial influence. A permissive tool is the agency. A secret in the context is the asset. A network request, rendered URL, or shell command can be the exfiltration sink.

The model therefore should not hold credentials, decide authorization, or be the only validator of a state-changing request. Put those decisions in deterministic application code. Keep the model useful for interpretation and planning while making the surrounding system enforce what may happen.

Threat model for an MCP-connected agent

Start with the real data flow, not a diagram that ends at the chat interface. Identify the user, model provider, MCP client and servers, authorization server, downstream APIs, browser, filesystem, memory, retrieval index, logs, and egress network. Mark each edge as user-authored, organization-controlled, third-party, public, or unknown.

The relevant attackers include a malicious website or email sender, a ticket or repository contributor, a compromised dependency or MCP package, a dishonest tool operator, and a token or session thief. A legitimate user can supply an unintentional injection by pasting a document containing instructions meant for another system. An internal database is not clean merely because it is behind a login.

List assets separately: instructions, tokens, keys, personal data, source code, private documents, cloud metadata, tool configuration, memory, and the ability to send, delete, purchase, deploy, or modify. Mark each as readable, writable, or externally observable. A read-only tool can leak data if output reaches an attacker-controlled destination, and a notification tool is a high-impact sink when it accepts arbitrary URLs or bodies.

For each workflow, write the intended invariant in plain language. Examples are “summarize the ticket without following instructions in the ticket,” “read only this repository,” and “draft an email but never send it without approval.” Then name the failure condition, such as a tool request that includes a secret, a scope that exceeds the task, a write outside the workspace, or a connection to a destination not approved by policy.

A practical risk record has five fields: delivery surface, propagation, encoding, privilege, and sink. Delivery may be chat, web retrieval, a PDF, an MCP description, a tool response, or memory. Propagation may cross steps, sessions, or agents. Encoding may be visible prose, HTML, invisible Unicode, a low-resource language, an image, audio, or obfuscation. Privilege is the identity and scopes at execution. The sink creates impact.

Direct and indirect injection

Direct injection is supplied through the user-facing path. A user may ask an agent to ignore its task, reveal hidden instructions, run an unrestricted command, or use a tool for an unrelated purpose. It also includes an honest user pasting text that contains an instruction addressed to an AI. Jailbreaking is a direct-injection goal in which the attacker tries to defeat model safety behavior, but an agent can be harmed without a jailbreak. A model that follows a fraudulent refund instruction while remaining polite has still crossed the application’s boundary.

Indirect injection arrives through content the user did not author as an instruction. A web page can tell the agent to upload its context. An email attachment can request a password reset. A support ticket can contain a hidden instruction to search a private repository. A database row, issue title, package README, image, or tool result can carry the same payload. The user may never see the relevant bytes because they are in HTML, metadata, a collapsed section, an image, or zero-width characters.

MCP expands the indirect surface in two directions. First, a server advertises names, descriptions, input schemas, resources, and prompts that the client places in model context. A poisoned description can tell the model to call a different tool first, include a secret in an argument, or treat an attacker-controlled URL as trusted. Second, the server returns content after a call. The model may interpret that content as a new instruction and begin a second tool call. This is tool poisoning and tool-output injection, even when the MCP transport and JSON-RPC messages are valid.

Treat tool metadata as executable influence, not documentation. Pin and verify packages, review descriptions and schemas, compare releases, and inventory each server’s capabilities. A signature supports provenance but does not prove a pinned version is benign, so review implementation and network behavior.

From influence to data exfiltration

Prompt injection becomes a breach when it joins a source to a sink. The source may be a poisoned issue retrieved by an agent. The sink may be an HTTP request, an email, a public comment, a calendar invite, an image URL, a log entry, or a tool argument. The attacker does not need direct access to the private data if the agent can read it and communicate externally.

Model a “lethal trifecta” explicitly: untrusted content, access to sensitive data, and an external or state-changing channel. Removing one leg is often more dependable than trying to detect every malicious sentence. A research agent can browse public pages in a logged-out, network-restricted environment. A document summarizer can read private files but have no network or send capability. A mail drafter can prepare a message without possessing the permission to send it.

Do not rely on output filtering alone. A secret can leave through a valid JSON field, a URL parameter, a tool’s diagnostic string, a generated image, whitespace or Unicode, or an apparently normal email. Enforce destination policy and information-flow rules before the sink. Redact secrets in logs and traces, and make outbound requests pass through an egress control that can deny private addresses, unknown domains, and unexpected methods.

MCP authorization and OAuth controls

For a protected HTTP MCP server, separate the protocol roles. The MCP server is the resource server, the client requests access on behalf of a resource owner, and the authorization server issues tokens. Follow the MCP authorization specification and its version-matched OAuth requirements instead of inventing a proxy flow.

Bind a token to the intended MCP resource. The specification requires the client to send the canonical server URI in the OAuth resource parameter in both authorization and token requests, and requires the server to validate that the token was issued for it. Use the Authorization header, never a query string. An MCP server must not accept or pass through a token issued for another resource. If it calls an upstream API, it should obtain and validate a separate upstream token. Token passthrough destroys audience separation, weakens rate limits and auditing, and can turn the server into an exfiltration proxy.

Use authorization code protection with PKCE, exact registered redirect URIs, HTTPS in production, and a cryptographically random, single-use state value. A proxy that dynamically registers clients must obtain consent for each client before forwarding a user to a third-party authorization server. Bind consent to the client identifier and requested scopes. A generic “this application has consent” cookie is not enough and can create a confused-deputy attack.

Treat discovery as input from the server, not as a trusted configuration file. Validate metadata URLs and authorization URLs with an allowlist of schemes and hosts. Reject javascript:, data:, and file: schemes. Protect server-side fetches from SSRF by blocking private, loopback, link-local, and cloud-metadata ranges, validating every redirect, and using an egress proxy. Avoid opening URLs through a shell. These checks cover ordinary MCP security risks that an injection can exploit, even though they are not themselves model defenses.

Use progressive scopes. Start with low-risk discovery and read access, request one additional scope only for the operation that needs it, and log the requested and granted subset with a correlation ID. Avoid *, all, and omnibus scopes. Claims do not replace server-side authorization on each tool and argument. Short-lived tokens, secure storage, refresh-token rotation for public clients, and careful logs limit token-theft impact.

Local MCP servers need a different boundary. A local process may run with the same user privileges as the client and can read files, use the network, or spawn commands. Show the exact startup command and arguments before a user approves a new server. Prefer stdio or a protected local IPC channel over an unauthenticated HTTP listener. Run the process in a sandbox or container with a minimal filesystem, network, and process policy, then grant additional directories or destinations explicitly.

Layered defenses that survive a bypass

Least privilege is the first blast-radius control. Give each agent only the tools needed for one workflow. Split read, draft, approve, and commit operations. Use short-lived, operation-specific credentials instead of a long-lived user token. Validate authorization again at execution time because the model’s plan, the user’s session, the tool description, and the resource may have changed since planning.

An approval gate should be a policy checkpoint, not a decorative confirmation. Require it for sending, deleting, publishing, paying, changing permissions, deploying, entering a new data class, or contacting a new destination. Display the exact action, arguments, identity, target, data fields, and side effect, not only a model summary. Recalculate the preview after approval and bind consent to an operation hash so later steps cannot substitute arguments. Rate-limit low-risk actions without training reviewers to approve opaque prompts.

Isolation limits what an injection can reach. Separate tenants, sessions, and memory stores. Give a browsing worker no private credentials. Place coding agents in a workspace sandbox with a narrow writable directory, denied-by-default network access, and a separate package identity. Combine process, filesystem, network, and browser isolation. Check container mounts, sockets, identity, egress, and escape paths.

Content provenance makes decisions explainable. Attach source, author, retrieval time, integrity status, and trust class to documents, tool descriptions, and results. Preserve it across summaries and hand-offs. Render external content as data with a visible, non-instructional boundary. Labels may guide the model, but application code must enforce permissions. Strip or canonicalize invisible characters at ingest and render boundaries, while accepting that other payloads remain possible.

Validate output in trusted code with strict schemas, allowlisted tools, typed arguments, URL and path checks, size and rate limits, and an explicit workflow state machine. Do not turn a second model’s safety opinion into authorization. A critic can flag behavior, but deterministic checks make the final decision. Record prompts, sources, metadata versions, arguments, policy results, approvals, and responses without raw secrets.

Adversarial evaluation and operations

Evaluate the whole agent loop, not just a chat prompt. Build cases for direct overrides, hidden HTML, quoted email, poisoned repository text, malicious MCP descriptions, tool output that requests a second call, cross-session memory writes, multilingual and encoded payloads, invisible characters, SSRF metadata URLs, token audience mistakes, and approval substitution. Include benign content that resembles an attack so that the team measures task completion and false positives together.

Test adaptively. Tell the red team which classifiers, labels, schemas, and approval rules are deployed. Ask it to vary wording, modality, timing, tool order, and data destinations. A static benchmark score can look excellent while an attacker who knows the defense finds a new route. Track attack success, unauthorized tool calls, sensitive bytes reaching sinks, blocked and allowed scope elevations, approval accuracy, time to detection, and time to revoke access. Keep regression cases after every model, prompt, server, policy, or dependency change. The AI agent evaluations guide can hold the workflow for this test program.

For architecture decisions, compare the controls with the production AI agent architecture guide. OAuth implementation details belong in MCP server TypeScript with OAuth.

Incident checklist

When an injection is suspected, stop the workflow and disable the smallest capability that contains it. Preserve the prompt, source, provenance, tool metadata and version, arguments, token metadata without values, policy decisions, approvals, destinations, and timestamps. Classify the path as direct, indirect, tool-borne, or memory-persistent, then search other tenants, indexes, queues, logs, and hand-offs.

Revoke credentials that may have entered context or logs, invalidate sessions and refresh tokens, and block the destination and affected server. Inspect downstream APIs for unauthorized reads, writes, messages, and OAuth grants. Check whether the executed arguments matched the approved preview.

After containment, remove poisoned memory or retrieval entries, restore trusted metadata, and add the path to regression tests. Record what the model saw, what policy allowed, what the user approved, and which control failed. Notify incident and privacy owners when data may have crossed a boundary. A later classifier result does not make the event harmless.

What protection cannot promise

Prompt injection remains an evolving control problem because current models do not enforce a formal separation between instructions and data. Training, classifiers, delimiters, provenance labels, output schemas, and human review can lower risk, but adaptive attackers can rephrase, encode, stage, or distribute an attack. Human approval can fail through fatigue, misleading previews, or a reviewer who lacks context. Sandboxes can be misconfigured, and a legitimate pinned package can still be compromised.

The right claim is not “the agent is immune.” The defensible claim is that a defined workflow has explicit trust boundaries, minimal capabilities, deterministic checks, observable decisions, tested failure modes, and a practiced response. Reassess those assumptions whenever a new MCP server, tool, memory provider, model, data source, or outbound channel is added. This article is guidance for that engineering work, not a certification or a security audit.

Sources

More