Prompt injection
Also known as LLM prompt injection, Indirect prompt injection
Definition
Prompt injection is an attack or unintended instruction that causes a language model to treat untrusted content as higher-priority guidance. It can redirect an application, expose data, or trigger tools in ways the developer did not intend.
Direct and indirect injection
In a direct injection, a user writes instructions designed to override the application's intended task. In an indirect injection, untrusted text arrives through a document, web page, email, or retrieval result. The model may follow that text even though it was not supplied as a trusted instruction.
Reduce the blast radius
Treat retrieved content and tool output as data, not policy. Limit tool permissions, validate arguments in application code, separate secrets from model context, and require confirmation for consequential actions. Output filters help, but they should not be the only control.
Test the whole workflow
A model response can look harmless while a later tool call causes damage. Create adversarial evaluation cases, trace each step, and measure both refusal behavior and useful task completion. Weave's trace context can help teams see where an injected instruction entered and what the agent did next.
How this relates to Weave
Weave helps teams inspect model and tool traces so a suspicious instruction can be connected to the input, route, tool call, and outcome that followed. Observability does not prevent injection, but it makes failures easier to reproduce and evaluate.
Explore Token intelligence