Prompt injection risk is often operational before it is technical.

01 / Research question

How should an enterprise assess prompt injection when the model is only one component of a larger application? The relevant attack surface includes user prompts, retrieved documents, webpages, emails, tool results, system instructions, memory, APIs, permissions, and downstream actions.

02 / Direct vs indirect injection

Direct prompt injection occurs when an attacker supplies malicious instructions through an interface that accepts input. Indirect prompt injection moves the malicious instruction into content the application later retrieves or processes, such as a webpage or document.

NIST’s Generative AI Profile explicitly describes both direct and indirect prompt injection and notes that indirect attacks can exploit LLM-integrated applications through retrieved content. OWASP likewise treats prompt injection as a core LLM application risk. citeturn0search50turn0search12

03 / Why the surrounding system matters

• A text-only chatbot may primarily expose integrity and confidentiality risks in its responses.
• A RAG system introduces retrieved content as another instruction/data boundary that must be evaluated.
• A tool-enabled assistant can turn model manipulation into actions against external systems.
• An agent with broad identity permissions can amplify a low-level prompt manipulation into a higher-impact workflow event.
• Persistent memory or shared context can make manipulation survive beyond a single interaction.

04 / Analysis

The key architectural issue is that natural-language instructions and untrusted data are processed by the same model interface. This creates ambiguity about which content should be treated as authoritative instruction and which should be treated as data. OWASP’s guidance also emphasizes that RAG and fine-tuning do not fully eliminate prompt-injection risk. citeturn0search12turn0search14

The practical consequence is that prompt-injection testing should follow the data and action path. A successful attack should be evaluated not only by the model’s changed response, but by whether the change crossed a trust boundary: disclosure of protected context, access to an unauthorized tool, modification of an external record, or influence over a consequential decision.

05 / What a Red Hawk lab scenario should measure

• Injection source: direct user input, retrieved document, webpage, email, tool output, or memory.
• Model behavior: instruction following, refusal behavior, context leakage, or unintended task switching.
• Boundary crossing: whether untrusted content influenced a privileged function or data source.
• Authorization: which identity and permissions were available to the AI workflow.
• Observability: whether the application logged the attack, tool call, data access, and resulting action.
• Containment: whether the application limited or blocked the unsafe action.
• Recovery: whether operators could identify, reverse, and investigate the event.

06 / Defensive design implications

No single prompt filter should be treated as the complete control. Stronger designs combine least-privilege tool access, explicit authorization checks, separation of instructions from untrusted data, output validation, action allowlists, monitoring, and human approval for high-impact actions. These controls address the application boundary rather than relying exclusively on the model to recognize malicious instructions.

OWASP’s current project also highlights related risks such as sensitive information disclosure and excessive agency, which are useful lenses for assessing what happens after a model is manipulated. citeturn0search10turn0search5

07 / Limitation

This brief describes an evidence-based assessment model. It does not claim that Red Hawk has achieved a particular attack success rate against a named commercial model or customer system. Original lab results should only be published after the corresponding controlled experiment has been executed and documented.