Defending Against Prompt Injection: A Practical Guide for LLM Apps

Prompt injection is the defining security problem of LLM applications, and the uncomfortable truth up front is this: you cannot fully solve it at the model layer. A language model processes its entire context — your system prompt, the user's message, a retrieved document, a tool's output — as one undifferentiated stream of tokens. It has no reliable notion of "these instructions are trusted and those are just data." So any text that looks like an instruction can become one. That's the whole vulnerability, and it's why the defense isn't a filter you bolt on — it's an architecture that assumes injection will sometimes succeed and ensures it doesn't matter much when it does.

Why it works (and why there's no clean fix)

Classic injection attacks — SQL injection, XSS — happen when data is mistaken for code. Prompt injection is the same bug at the semantic layer: in an LLM, instructions and data share one channel. You can ask the model to "only follow instructions in the system prompt," but the model is a probabilistic text predictor, not an interpreter with a privilege boundary — a sufficiently convincing injected instruction can win. Researchers keep finding new bypasses; defenders keep patching phrasings. Anyone selling a complete fix is selling you a false sense of security. Prompt injection sits at LLM01 in the OWASP Top 10 for LLM Applications precisely because it's foundational and unsolved.

The dangerous variant: indirect injection

The injection you should fear most isn't the user typing "ignore your instructions." It's indirect (second-order) injection, where the payload rides in on content your system reads as part of its normal job:

• a poisoned passage in a retrieved RAG document,
• instructions hidden in a web page an agent browses,
• a crafted email or ticket it summarizes,
• the output of a tool it called.

For an agent with tools, every external source is an attack surface, and the user is often unaware the payload exists. This is why agentic systems raise the stakes: an injected instruction can become a real action — sending data, calling an API, spending money.

Defense in depth

Because you can't stop injection at the door, you limit what it can do once inside. In rough order of leverage:

1. Least privilege — the strongest control

Give the model the minimum tools and permissions to do its job, and nothing more. An agent that can only read can't be made to write. Scope every credential and tool tightly, so a successful injection inherits a small, safe surface rather than the keys to everything. This single principle does more than any input filter.

2. Human approval for high-impact actions

Put a person in the loop for anything irreversible or high-impact — sending money, deleting data, emailing customers, changing permissions. An injection that can only propose such an action, not execute it, is largely defanged. (See Production Tool/Function Calling for wiring approvals into the loop.)

3. Trust boundaries on all external content

Treat retrieved, tool, user, and web content as untrusted data, never as trusted instructions. Don't blindly concatenate it into the same instruction space; mark it, structure it, and minimize how much of it the model treats as directive. Delimiters and clear roles help at the margin — they are not a guarantee, so don't rely on them alone.

4. Input and output guardrails

Layer scanners that catch known injection patterns on the way in and validate outputs on the way out — schema conformance, policy checks, PII/secret leakage, off-topic or unsafe content. Tools like LLM Guard and NeMo Guardrails implement these as input/output rails. Treat them as defense in depth, not a wall: they raise the cost of an attack, they don't end it.

5. Keep secrets out of reach

Assume the model's context — including your system prompt — can be exfiltrated (system-prompt leakage is LLM07). Don't put credentials, API keys, or sensitive data where the model can read and leak them. What isn't in the context can't be injected out of it.

6. Sandbox and validate tool execution

Run tools with constrained permissions and validate their outputs before the model acts on them — both because tool output is an injection vector and because a compromised tool shouldn't get free rein.

Test it like an attacker

Defenses rot as attacks evolve, so make red-teaming continuous, not a one-time audit. Probe your own system with injection and jailbreak payloads — directly and via the indirect channels (poisoned docs, tool output) — and gate releases on the results. promptfoo automates adversarial red-teaming for prompt injection and jailbreaks; the Red Team LLM command runs a structured probe and the prompt-injection-auditor audits the app's trust boundaries and blast radius.

Putting it together

Accept that prompt injection can't be eliminated, then make it not matter: least privilege, human approval for high-impact actions, strict trust boundaries on all external content, input/output guardrails, secrets kept out of context, sandboxed tools — and continuous red-teaming. The goal isn't a model that can't be fooled; it's a system where fooling the model buys an attacker almost nothing. For the broader agentic threat landscape this sits inside, see Securing AI Agents: The OWASP Agentic Top 10 in Practice.