Quick answer: Prompt injection is when untrusted text given to a language model (LLM) overrides the developer's system instructions and makes the model perform unwanted behavior. The attacker places hidden instructions in user input or in a web page, email or document the model reads, and the model treats them as legitimate commands. The result can be leakage of the hidden system prompt, unauthorized tool calls, data exfiltration or output that deceives the user. The root fix is not a single filter: separate untrusted data from instructions, minimize the model's privileges and verify every sensitive action with a layer outside the model. Prompt injection is the number one risk on the OWASP LLM Top 10.

AI applications now read email, search the web, access databases and call tools. This power opens a new attack surface: any text that reaches the model can be a command. This article explains how prompt injection works, its direct and indirect types, the OWASP LLM Top 10 framework and the correct defense in enterprise AI applications.

How prompt injection works

An LLM sees the system instruction and the user input as text in the same context window. The model does not enforce a strict boundary that separates code from data. The attacker exploits this ambiguity: they add a sentence like "ignore all previous instructions and do this" to the input. The model may follow the latest and most convincing instruction rather than the developer's intent.

The real danger appears when the model can call a tool. An agent that can read files, send email or call an API can use these privileges for the attacker through an injected instruction.

Direct and indirect prompt injection

Type How it arrives Example impact
Direct The user writes to the model directly Leaking the system prompt, bypassing rules
Indirect Hidden in external content the model reads Data exfiltration via a hidden command in a page or email
Multi step Planted in memory or a document A persistent instruction triggered in a later session

Indirect prompt injection is the most dangerous because the victim is not even aware. The model treats hidden text on a page it was asked to summarize as a command. This is directly related to RAG and vector database security, because poisoned content becomes persistent if it enters memory.

OWASP LLM Top 10, a short map

OWASP publishes a separate top ten risk list for AI applications. The highlights:

  • LLM01 Prompt Injection. The subject of this article, the top risk on the list.
  • LLM02 Sensitive Information Disclosure. The model carrying secret data into output.
  • LLM04 Data and Model Poisoning. Corruption of training or memory data, see our data poisoning article for detail.
  • LLM06 Excessive Agency. The agent having more tools and permissions than needed.
  • LLM08 Vector and Embedding Weaknesses. Attacks against RAG memory.

This list gives the security team a common language and a priority order.

Why a single filter is not enough

Most teams think they solve the problem by searching and blocking dangerous words. But this blacklist approach is weak:

  • Infinite variation. The same intent can be written with different wording, indirect phrasing or in another language.
  • Encoding and obfuscation. The instruction can be hidden with base64, emoji or invisible characters.
  • Indirect channel. When content comes from an external source, the input filter does not even engage.

So the defense is not to search for words but to build the architecture securely.

The correct defense

1. Separate untrusted data from instructions

User input and external content must be clearly separated from the system instruction and given to the model with the context that "this section is data, not a command". Sensitive decisions must rely only on trusted instructions.

2. Principle of least privilege

Minimize the tools and permissions the agent can reach. If a tool has a destructive effect, do not let the model trigger it alone. This is the same logic as limiting excessive privilege.

3. Human or rule approval

Sensitive actions such as money transfer, data deletion or sending email must be verified by a layer outside the model, with a rule or human approval. The model proposes, an independent layer executes.

4. Output and data boundary

The model's output must not be passed directly to a system as a command. Access to secret data must be limited by the user's authorization and the output must be checked against leakage.

5. Local and isolated execution

Instead of sending enterprise data to external APIs, running a local LLM on your own server reduces the data exfiltration surface and the risk of employees leaking data to AI.

AI application security with KAOS

DSET's local AI security engine KAOS tests LLM based applications for prompt injection and OWASP LLM Top 10 risks. KAOS tries direct and indirect injection vectors, detects the agent's overprivileged tool calls and verifies in a controlled way whether an injection actually triggers a sensitive action. It reports only proven findings without false positive noise. Because KAOS runs fully offline, enterprise data does not leave during testing, which matters for KVKK compliance.

Frequently asked questions

Are prompt injection and jailbreak the same thing? Close but not the same. Jailbreak focuses on bypassing the model's safety rules to produce forbidden content. Prompt injection is broader: any untrusted text the model reads overriding the system instruction to force unwanted action. Jailbreak can be seen as a subset of prompt injection.

Is a good system prompt enough? No. A strong system prompt helps but is not trustworthy alone. The model can override the instruction with a sufficiently convincing injection. Real defense is in the architecture: privilege limitation and an approval layer outside the model.

Does using a local model solve prompt injection? A local model reduces the data exfiltration surface and the privacy risk but does not solve prompt injection alone. Injection is independent of where the model runs. Privilege limitation and an approval layer are still needed.

Sources

To test your AI application against prompt injection and OWASP LLM Top 10 risks, contact DSET. We provide offline, KVKK compliant security testing with KAOS from our Ankara Hacettepe Teknokent laboratory.