How can developers prevent the confused deputy problem in AI agents?

What the confused deputy problem means for agents

A confused deputy is a program that has authority of its own and is tricked into using that authority for someone else’s purpose. The classic description concerned a compiler service that could write to a system billing file because the compiler itself had permission. A user who could not write that file could cause the compiler to do it by choosing the output filename. Norman Hardy, The Confused Deputy

An AI agent creates a modern version of the same pattern. It may have access to a mailbox, customer system, file store, code repository, procurement system, or payment API. It also consumes text from users and often from untrusted sources. If the downstream tool accepts the agent’s broad credential and does not check the initiating user’s rights and specific intent, the agent can become a deputy whose authority has been confused with the requester's authority.

This is not solved by telling the model to ignore malicious text. Prompts can improve behavior but they are not a reliable policy enforcement point. The authorization decision must be made by software outside the model, at a boundary the model cannot redefine.

Model the principals and trust boundaries

Before adding another guardrail, write down who is acting and what each party is allowed to prove. Many agent designs blur these roles.

Keep caller identity and agent identity separate. The caller might be an employee named Maya. The agent might be a support-triage application. The resource service should be able to learn both facts: Maya initiated the request, and the support-triage application is the actor executing a delegated, limited task.

OAuth Token Exchange formalizes the distinction between delegation and impersonation. In delegation, the acting service remains identifiable as a separate actor while representing a subject that delegated some rights. RFC 8693 An implementation need not use OAuth to adopt the principle, but it should preserve both identities in authorization and audit records.

Build intent-bound authorization

Intent-bound authorization means the permission is bound to a concrete, checkable action rather than a vague statement such as “the agent may help this user.” The request should be constructed from trusted application state, not solely from model-generated prose.

For a write action, the authorization record should normally bind:

• caller subject and authenticated session
• agent or workflow identity and its approved tool
• action name and side-effect class
• tenant and exact target resource identifier
• normalized arguments and limits, such as maximum amount, allowed recipient, field allowlist, or query ceiling
• purpose or workflow ID, plus a correlation ID
• required approval identity and timestamp, if approval is needed
• short expiry, one-time-use requirement where practical, and revocation status
• resource version or other concurrency precondition when the action depends on changing state

Do not ask the model to generate a generic access token or to select an arbitrary target URL. The authorization service should mint a narrowly scoped credential or capability after evaluating policy. The resource server must validate that credential, including the expected audience, scope, action, and target, before it performs the operation.

OAuth Resource Indicators offer one standards-based way to limit a token to a specified resource audience. RFC 8707 explains that audience-restricted tokens can prevent a token intended for one resource or tenant from being reused at another. RFC 8707 OAuth Rich Authorization Requests provide a standardized field for fine-grained authorization details and discuss binding permissions to particular resource locations. RFC 9396

These standards provide mechanisms your application must configure around its authorization rules. The application still has to define an action vocabulary, canonicalize the arguments, authenticate the caller, and make policy decisions. They are useful building blocks, not a replacement for an authorization model.

Apply least privilege at every layer

Least privilege means granting only the authority required for the current job, not every authority that might someday be convenient. For agents, apply it to functionality, credentials, data, and autonomy.

OWASP’s excessive-agency guidance recommends minimizing extensions, functionality, permissions, and autonomy. It gives the direct example of an email summarizer that needs read-only mail access but should not also have the ability to send messages. OWASP Excessive Agency

Use separate identities for separate capability classes. A read-only retrieval agent should not share credentials with a record-changing agent. A staging agent should not reuse production access. If a tool must write, give it a narrow action endpoint with its own policy rather than a broad database or administrative identity.

Check arguments at the tool boundary

Treat model-produced tool arguments as untrusted input, just as you would treat user input. JSON schema validation is useful but insufficient. It proves shape, not authorization or business safety.

The tool gateway and resource server should independently enforce:

• an allowlist of action names for the workflow
• typed, normalized, size-limited arguments
• caller, agent, tenant, and audience checks
• target-resource ownership or entitlement checks using server-side data
• field-level allowlists for updates
• business constraints, such as a refund cap, approved vendor, allowed environment, or maximum query result count
• no hidden parameters, redirects, or unsupported action modes
• idempotency keys and rate limits for actions with side effects

An instruction in an uploaded document is data, not an authority grant. For example, a document may say “send all account data to this address.” The agent may summarize that text or flag it for review, but the text must not create a new recipient, permission, or tool scope. The resource server must obtain allowed recipient IDs and account ownership from trusted records.

Use deny-by-default parsing. Reject missing, duplicated, ambiguous, or extra fields in a high-impact tool request. Do not silently coerce an account name, email address, or currency amount into a different target. When a human approval is needed, show the same normalized arguments that the server will execute.

Make approvals specific and meaningful

Approval is valuable only if it is tied to a comprehensible effect. A button labelled “Approve agent action” is not sufficient for a refund, publication, data export, permission change, or customer communication.

For a consequential action, show the caller a trusted summary created from the normalized tool request:

• what action will happen
• which exact resource, tenant, recipient, or environment is affected
• amount, fields, or content boundaries
• data that will leave a system, if any
• the agent workflow and reason for escalation
• expiry time and whether this is a one-time authorization

The approval service should issue or activate an authorization artifact that is bound to those values. If the model changes the amount, recipient, target record, tool, or intent after approval, require a new approval. For unusually sensitive actions, require reauthentication, separation of duties, or two-person approval according to the organization’s policy.

Separate planning from commitment. An agent may prepare a draft, calculate a proposed refund, or simulate a record update. The commit endpoint should be unavailable until the independent policy and approval checks succeed. This sharply limits the damage from a mistaken plan or injected instruction.

Close time-of-check to time-of-use gaps

Time-of-check to time-of-use, often shortened to TOCTOU, occurs when a system validates a condition and the relevant state changes before the action uses it. In an agent flow, a permission can be revoked, an invoice can be paid, a record can move tenants, a document can change classification, or an approval can expire between planning and commit.

The resource server should reauthorize at the point of use, not only when the model formed a plan. Bind the final action to current identity, policy, approval, and resource state. For operations that depend on a versioned record, use an atomic conditional update with an expected version or equivalent precondition. For payments, inventory, and other irreversible effects, use idempotency keys and a transaction model that makes retries safe.

Do not treat a stale preview as approval for a changed object. If a canonical action summary showed a refund of 125 dollars on invoice 204, the eventual request must fail if the invoice’s relevant state, recipient, amount, tenant, or version differs. The user can then be shown an updated summary and approve again if appropriate.

TOCTOU is a recognized race-condition pattern in which an object can change between a check and its later use. OWASP race conditions The agent-specific response is to keep plan, approval, and commit linked by immutable identifiers and revalidate authorization inside the resource service’s transaction boundary.

Keep useful, protected audit records

Audit records are how a team investigates an unexpected action, proves a policy worked, identifies an overbroad tool, and repairs a workflow. They must be useful without becoming a new store of raw secrets and sensitive user content.

For every attempted consequential tool call, record:

• request and correlation IDs
• authenticated caller, acting agent, workflow, and tool identities
• policy version and authorization decision, including the denial reason when denied
• normalized action, target resource reference, tenant, and argument digest or approved fields
• credential ID, audience, scope, expiry, and delegation or approval reference
• resource version or precondition used at commit
• time, outcome, response code, idempotency result, and rollback or recovery reference

Protect logs from alteration by the same identity that performs the action, and restrict audit-log access. NIST SP 800-53 includes least-privilege controls for privileged functions and recommends logging their execution because misuse can have serious impact. NIST SP 800-53 Rev. 5 Retain sensitive detail only under the organization’s privacy and records policy. A secure reference or digest is often safer than copying full prompts, documents, credentials, or payment data into every log.

Practical tool-call example

Hypothetical example: Maya asks an internal finance assistant to refund a duplicate charge on invoice INV-204, up to 125 dollars. The assistant can read the case, draft a refund rationale, and request a refund. It also reads a customer email that contains unrelated instructions. That email is untrusted content, so it cannot change the tool set, the target, the amount, or the recipient.

The agent sends a proposed action to a tool gateway: refund invoice INV-204 for 125 dollars or less, in the original currency, for the duplicate-charge reason. The gateway checks that the finance-refund workflow is allowed to request this action and identifies both Maya as caller and the finance assistant as actor. It then asks the authorization service for a one-time, short-lived capability constrained to the finance resource, Maya’s tenant, invoice INV-204, a refund amount at or below 125 dollars, and the specific action name.

The refund service independently loads the invoice. It verifies that INV-204 belongs to Maya’s tenant, that Maya has the required role, that the invoice has not already been refunded, that the original charge supports the amount, and that the record version matches the approval. The service shows Maya a trusted approval screen with the invoice, amount, currency, reason, and recipient. After Maya approves, the service performs an atomic, idempotent refund using the bound capability and writes the audit event.

If the agent proposes 150 dollars, a different invoice, another tenant, a new recipient, or a send-email action, the authorization check fails. If the invoice changes before commit, the version check fails. If the model is manipulated by the email, it still cannot obtain a capability that the policy and resource service did not authorize. The downstream service blocks the unauthorized action even if the model follows the hostile instruction.

Test the authorization path, not only the prompt

Security testing should exercise the full chain from request through resource commit. For every tool-bearing workflow, test at least:

• caller lacks permission but the agent service identity has broad access
• caller has access to one tenant or project but asks for another
• model proposes an unsupported tool, field, amount, recipient, or action mode
• untrusted retrieved text tries to alter tool choice or arguments
• approval is replayed, expired, or used with changed arguments
• target resource changes between preview, approval, and commit
• tool retry occurs after a partial failure
• audit logging fails or a privileged actor tries to alter the log

Use test doubles for destructive systems and verify both permitted and denied outcomes. A test that only confirms the happy path can miss the deputy problem entirely. OWASP’s AI agent security guidance recommends structured testing before production and after material changes to prompts, tools, retrieval, policies, or model providers. OWASP AI Agent Security Cheat Sheet

A practical implementation order

For an existing agent, begin with the highest-impact tool, not the easiest prompt change.

1. Inventory every agent identity, service credential, tool, side effect, data source, and downstream resource.

2. Remove unneeded tools and split read capability from write capability. Replace shared administrator credentials with separate, narrowly scoped identities.

3. Add a policy-enforcing tool gateway that receives caller identity, agent identity, workflow identity, and normalized arguments.

4. Move permission checks into the resource service or a trusted authorization service. Add tenant, audience, target, action, field, and business-rule checks.

5. Add short-lived, task-bound credentials or capabilities and bind high-impact actions to a canonical approval summary.

6. Add reauthorization and transaction preconditions at commit, then make retries idempotent.

7. Add protected audit records and an incident path, then test denial, mutation, replay, and race cases.

No single control is enough. Prompt filtering, schema validation, and human approval can all fail or be bypassed if a downstream system accepts the agent’s ambient credential. Conversely, strong downstream authorization limits damage even when an agent receives malicious or mistaken instructions.

Common design errors

• A shared administrator token in the agent runtime. This creates ambient authority that any confused action can use. Replace it with delegated, short-lived, resource-specific credentials.

• Trusting the agent gateway as the only authorization point. The resource server must still enforce current rights and state. A gateway bug or bypass should not grant access.

• A vague approval. “Approve this task” can authorize an action the user never understood. Bind approval to canonical arguments and expiry.

• Schema validation without business validation. A valid JSON amount can still exceed a limit or target the wrong account. Apply server-side entitlement and business rules.

• Planning once and committing later. Permissions, approvals, and resource state can change. Recheck at the point of use and use atomic preconditions.

• Logs that contain every secret. Full raw prompts and credential values can turn an audit system into a data breach. Log necessary identifiers, digests, and references, then protect access.