Traditional application security draws a fairly clear line: user input is untrusted, application code and its own data are trusted. AI systems blur that line, because a model treats everything in its context window as just text — a retrieved document, a tool result, or a piece of conversation history can all carry instructions the model may act on, whether or not the application intended them as instructions.

Input boundary

Where user input first enters the system — validated before it reaches the model or gets embedded into a prompt.

Retrieval boundary

Where retrieved content enters the model's context — this content should be treated as data to reason about, not instructions to follow, and access-filtered by permission metadata.

Tool boundary

Where a model-requested action actually executes — this is where least privilege and approval checkpoints matter most, since it's the point where a decision becomes a real-world effect.

Output boundary

Where the model's response leaves the system — validated before it's shown to a user, stored, or used to trigger another action.

Illustrative pseudocode showing where checks belong relative to the model call, not a complete security implementation:

def handle_request(user_input, tenant_id):
    validate_input(user_input)  # input boundary

    retrieved = retrieve(user_input, tenant_filter=tenant_id)  # retrieval boundary:
    context = mark_as_untrusted_data(retrieved)                # tag, don't treat as instructions

    response = model.generate(system_prompt, context, user_input)

    if response.requested_tool_call:
        if not tool_is_allowed(response.tool_name, tenant_id):  # tool boundary
            raise PermissionDenied()
        if requires_approval(response.tool_name):
            await human_approval(response)
        result = execute_tool(response.tool_name, response.arguments)

    return validate_output(response)  # output boundary

• Scope tool access per use case — an agent handling customer questions shouldn't have the same tool access as one handling account administration.
• Scope data access per request — retrieval should be filtered by the requesting user's or tenant's actual permissions, not the broadest access available to the system.
• Separate read and write capabilities — a tool that can read data is a much smaller risk surface than one that can also modify or send something.
• Keep secrets out of prompts entirely — API keys, credentials, and internal identifiers shouldn't be constructed into a prompt where a model could echo them back.

• Treating a strong system prompt as a security control

  Prompt instructions influence behavior but are not a reliable security boundary against adversarial input.

• Trusting retrieved content or tool results by default

  Anything the model reads that wasn't written by the application itself should be treated as untrusted data.

• Granting broad tool permissions for convenience

  Excess permissions turn any single injection or mistake into a much larger incident than it needs to be.

• Validating only input and not output

  A model can produce unsafe or policy-violating output even from a reasonable, fully validated input.

• Logging full prompts and responses without redaction

  Logs are a common place for sensitive user data or secrets to leak if captured indiscriminately.

• No audit trail for tool calls or data access

  Without logging what an agent accessed or executed and why, incident response and compliance both become far harder.