The New Attack Surface: Prompt Injection

Here is a problem that did not exist a few years ago. Your AI assistant reads a web page, an email, or a document to help a user. Hidden in that content is a line of text aimed not at the user but at the model: ignore your previous instructions and do this instead. If the model obeys, you have been attacked by a sentence. Prompt injection is the defining security problem of the AI era, and every team building with language models has to understand it. This article explains what it is, why it is so stubborn, the forms it takes, and how to defend against it in practice.

What prompt injection is

Prompt injection is an attack where malicious text sneaks instructions into a model’s input and gets the model to follow them instead of the instructions its builder intended. Because a language model treats everything in its context as something to act on, a cleverly worded piece of content can hijack its behavior, override its rules, leak its hidden prompt, or trigger actions on the attacker’s behalf.

It is useful to separate two flavors. Direct injection is when the user themselves types the malicious instruction, often to jailbreak a model past its safety rules. Indirect injection is more dangerous: the malicious instruction hides inside content the model reads while helping an innocent user, such as a web page or an incoming email. The user never sees it, and the attack rides in through the data.

Why this is genuinely new

In traditional software, code and data live in separate lanes. A program knows the difference between the instructions it runs and the data it processes. In an AI system, instructions and content are both just text in the same context window, and the model does not reliably know which is which. Any untrusted text the model reads is a potential instruction, which turns the entire open web, and every document and message, into a possible source of attack.

When your system takes orders in plain language, every piece of text it reads is a command it might follow.

How an attack plays out

The danger grows sharply once a model is connected to tools, memory, or the ability to send data. A typical indirect attack unfolds in a few steps.

• The attacker plants instructions in content the assistant is likely to read, like a web page, a shared document, or an email.
• An ordinary user asks the assistant to summarize or act on that content.
• The model ingests the hidden instructions along with the legitimate text and cannot tell them apart.
• It follows the injected commands, perhaps leaking private context, calling a tool, or sending data to an attacker-controlled destination.

The worst outcomes combine injection with capability. A model that can read your email and also send email can be told, by a message it reads, to forward your inbox somewhere. The injected sentence did nothing on its own; the tools gave it teeth.

How to defend

There is no single fix, and anyone selling you one is overselling. Prompt injection is managed, not solved, through layers that assume the model can be fooled and limit the damage when it is.

Practical defenses

• Treat all external content as untrusted data, never as trusted instructions, and separate it clearly in the prompt.
• Apply least privilege: give the model only the tools and access it strictly needs, so a hijack cannot reach anything dangerous.
• Keep a human in front of consequential actions, so a crafted sentence cannot send money, delete data, or email on its own.
• Constrain outputs and tool calls, validating them before anything acts on them rather than trusting the model’s word.
• Monitor and log, so unusual tool use or data access is caught quickly rather than discovered after the fact.
• Red-team your own system, actively trying to inject it, before an attacker does.

What does not work on its own

Telling the model to ignore injected instructions helps a little but is not reliable, because the next clever phrasing can talk it out of that rule too. Filtering for known bad phrases catches yesterday’s attacks, not tomorrow’s. These measures are worth having as layers, but none of them is a wall, and treating any of them as a complete solution is how systems get breached.

Frequently asked questions

What is prompt injection in simple terms?

It is tricking an AI model by hiding instructions in the text it reads, so it follows the attacker’s commands instead of its builder’s. Because the model treats all text in its context as something to act on, a well-crafted sentence can hijack what it does.

What is the difference between prompt injection and jailbreaking?

Jailbreaking is one kind of direct prompt injection, where a user crafts input to get past a model’s safety rules. Prompt injection is the broader category and includes indirect attacks, where the malicious instruction hides in third-party content the model reads while helping someone else.

Can prompt injection be completely prevented?

Not with any single technique available today. The realistic goal is to contain it: assume the model can be manipulated, limit what it is allowed to do, and keep humans in front of consequential actions so a successful injection cannot cause serious harm on its own.

Does prompt injection matter if my AI has no tools?

It matters less, but it is not harmless. Even a read-only assistant can be made to leak its hidden system prompt, produce content that embarrasses your brand, or give users deliberately wrong answers. The stakes rise sharply the moment the model gains access to tools, data, or actions.

Key takeaways

• Prompt injection hides instructions in text a model reads, hijacking its behavior.
• It is new because AI systems cannot cleanly separate trusted instructions from untrusted data.
• Indirect injection, hidden in third-party content, is the most dangerous form.
• The risk multiplies when a model is connected to tools, memory, or the ability to send data.
• There is no single fix: defend with least privilege, human approval, validation, monitoring, and red-teaming.

Prompt injection is managed through layered caution, least privilege, and the assumption that the model can be fooled. In AI security, that assumption is not pessimism; it is the starting point. If you are building an AI feature that touches real data or takes real actions, designing it so a hostile sentence cannot do damage is exactly the kind of applied security work we do at Vesper Labs.