Prompt Injection
Prompt Injection explained
An assistant is asked to summarise a document. The document also contains an instruction addressed to the AI system. The key distinction is that the is task material, not new permission to change the task. Failure to maintain that boundary can produce an incorrect response or an unauthorised action.
The issue extends beyond text composition. Tool access can expose data and actions too. Possible consequences therefore depend on what the system can actually read, change or share.
A useful implementation separates the request from outside content and limits access to what the task requires. Enforce permissions in the system. Additional checks and approvals can constrain particular consequences; no single prompt wording is a universal solution.
Assess the application with controlled examples in a suitable test environment. Check whether it still fulfils the permitted task and prevents unwanted actions. A strong defence rate on a limited test set does not establish protection against every future variation.
Examples
Hypothetical application
A research assistant receives a test page containing factual product information and a conflicting instruction addressed to the system. It should use the facts for its task without treating the outside instruction as a request. Assessment covers its answer and possible actions, not just a warning message.
Key Points
- Outside content must not independently expand the task.
- Consequences depend on data access and permitted actions.
- Combine safeguards with technical permission enforcement.
- Document test coverage and limits alongside results.
Practical application
Identify the external an AI application processes and the actions it can trigger. Define boundaries and assess compliance through controlled test cases. Record observed failures and the effects of specific mitigations.
Useful measures
Task adherence in tests
Whether the permitted task is fulfilled despite manipulated test content.
Unauthorised actions
Observed attempts or completed actions outside defined boundaries.
Coverage and false positives
Content types and situations assessed, plus legitimate tasks blocked unnecessarily.
Common mistakes
- Checking only direct user input while overlooking outside content.
- Equating a protective prompt statement with enforced access control.
- Treating one successful example as complete security evidence.
Sources and context
- OWASP: LLM01 Prompt Injection
Technical context on direct and indirect prompt injection and the limits of mitigations.
- Anthropic: Prompt injection defenses in browser use
Research on defending against malicious instructions in browser content; not evidence of complete immunity.
Frequently Asked Questions about Prompt Injection
The influencing instruction reaches the application through material such as a webpage or file rather than as the user’s direct request. The application should process that material without treating it as higher-authority instructions.
No. Retrieved content can also contain manipulative instructions. Supplying sources, checking factual accuracy and preserving task boundaries are different jobs.
That may be one part of a defence. Also use bounded technical permissions, appropriate checks and assessment of the complete workflow.
Related links
Loading related terms…
All TermsArticles about Prompt Injection
AI with Brand DNA: Why Generic Bots Are a Brand Risk
Off-the-shelf AI assistants can dilute your brand. Learn how strategic calibration, guardrails, and red-teaming can transform a generic bot into a powerful, on-brand ambassador that positively impacts business outcomes.
Grok Bot Skills: How to Truly Scale AI in Your Marketing
Reusable task instructions help teams organise AI work consistently. Learn how to connect briefs, data, quality checks and version control, and measure the benefits in your own workflow.
Prompt Ops: The Operating System for AI in Marketing
The uncontrolled use of AI prompts leads to chaos. Prompt Ops provides a structured approach to manage prompts like software, ensuring efficiency, quality, and scalability in marketing.