A run ends after the model proposes delete_file, but before the tool result is stored. The client later resumes the conversation, reorders part of the transcript, and sends the unresolved structured call back to the runtime. If the orchestrator rebuilds execution state from message history, the old proposal can be dispatched under a new user, policy version, or approval context without the model making a fresh decision.
No prompt injection is required in that path. The stale call enters after reasoning and may avoid the model hooks or confirmation flow that engineers assumed every operation would cross. The security question belongs at dispatch: can the runtime prove that this exact call was produced and approved for the current run, principal, tenant, resource, policy, and budget?
Conversation history is useful evidence and model context. It is not authoritative control state. Server-issued call records, approvals, idempotency data, and lifecycle transitions need their own integrity and ownership rules; every dispatch route must re-evaluate current authority before the tool reaches a protected effect.
Key Takeaways
- Structured agent history can carry unresolved tool calls, call identifiers, tool results, approvals, and state transitions; it must be validated as executable input rather than trusted as a transcript.
- History supplied by a client, restored from storage, imported from another run, or transformed by middleware needs explicit provenance and cannot independently authorize a tool.
- Sanitization and message-sequence repair reduce ambiguity, but sensitive tool handlers must still authenticate and authorize every invocation at dispatch time.
- A valid tool name and schema-compliant arguments do not prove that the call belongs to the current principal, tenant, run, approval, or policy version.
- Autonomous security testing should mutate message order, unresolved-call state, provenance, hooks, retries, and direct dispatch paths while verifying protected side effects.
- The most useful evidence shows whether an unauthorized tool effect occurred, which enforcement layers were skipped, and whether the same sequence remains blocked after remediation.
Why History Has Executable Semantics
Most chat histories contain roles and content. Agent histories add structured events:
user message
assistant reasoning output
assistant tool call: tool=T, arguments=A, call_id=C
tool result: call_id=C, result=R
assistant continuation
The orchestration runtime interprets this structure. It may discover a pending assistant tool call and dispatch it. It may correlate a tool result with a call ID. It may pass the result back to a model for another decision. It may resume a run after a process restart. Those behaviors make history part of a state machine, not merely text shown in a user interface.
The same semantics create risk when history crosses a trust boundary:
- A browser submits prior messages to a stateless API.
- A user edits or imports a conversation.
- A database record is restored after a crash.
- One agent delegates a sub-history to another.
- A support operator replays a failed run.
- A queue retries a partially completed step.
- A framework compacts or sanitizes malformed history.
- An MCP client or server transports structured tool-use messages.
At each boundary, the runtime must distinguish descriptive history from authoritative execution state. "This transcript says the assistant called delete_project" is not the same as "the current trusted orchestrator has authorized and scheduled delete_project for this principal."
The Dangerous Category Error
Systems often make a category error by serializing internal state into a message format and later trusting the serialized form as if it remained internal. Once the history is sent to a client, stored in a mutable table, copied into a support ticket, or accepted through an API, its integrity and provenance assumptions change.
Signatures or authenticated storage can prove that bytes were not altered after issuance. They still do not prove that an old call is valid now. The principal may have changed, the tenant membership may have been removed, the tool may require new approval, the policy may be stricter, or the underlying resource may now belong to someone else. Integrity is necessary for trusted resumption, but freshness and authorization remain live decisions.
A Tool Call Is a Capability Request
A useful model treats every tool call as a request for a capability:
Principal P requests action T with arguments A
against resource set R in tenant N,
within run X and policy version V,
under approval and budget context G.
Dispatch is allowed only if the runtime can establish all of those bindings from trusted state. The model can propose T and A; it cannot grant itself the capability. A history message can describe a previous proposal; it cannot reactivate authority. Tool metadata can inform planning; it cannot substitute for server-side enforcement.
This distinction matters for both local functions and remote MCP tools. A tool named read_file may reach a constrained workspace or an entire host. send_email may send only drafts or may contact arbitrary recipients. query_customer may enforce tenant filters or accept a raw account ID. Names and descriptions are hints about intended behavior. The handler and its downstream services define actual authority.
Dispatch Invariant
The core invariant can be written as:
execute(call) only if:
call was derived through a trusted current-run transition;
the current authenticated principal may invoke the tool;
the principal may act on every resolved target resource;
arguments satisfy syntax and semantic policy;
required approval is valid for this exact call;
limits and environment guardrails permit execution;
replay or duplicate execution cannot repeat a protected effect.
No single message field proves the invariant. The dispatcher needs trusted context and the tool handler needs to enforce resource authorization close to the effect.
Where History Integrity Breaks
Several failure patterns can expose unresolved tool calls or convert descriptive data into commands.
Truncation Around an Unresolved Call
Imagine a history ending with an assistant tool request followed by a user message that cancels or changes the task. A sanitizer removes the trailing user message because it considers the sequence malformed, leaving the assistant call as the final unresolved event. If the runtime resumes by dispatching pending calls, cleanup has changed behavior rather than merely repairing syntax.
The lesson is not that every sanitizer is vulnerable or that unresolved calls are always executed. It is that transformations must preserve security meaning. Dropping a message can expose an earlier executable state. A safe repair may need to invalidate the whole pending turn, convert the call into non-executable history, or require a fresh model decision.
Call and Result Splicing
Call identifiers correlate proposed actions and results. If IDs are accepted without run binding, an attacker may inject a result from another conversation, satisfy a pending call without execution, or attach a pending call to an unrelated result. This can corrupt model context and authorization state even when no direct tool invocation occurs.
IDs should be unpredictable or scoped, but entropy alone is not authorization. Store (run, turn, call, principal, tenant, tool, argument digest, status) as a relationship. A result is accepted only for the exact pending call in that trusted run.
Cross-Principal Resume
A run created by one user may be shared, transferred, or opened by another. If resumption relies only on a run ID or session handle, pending calls can execute under whichever server credential happens to be available. Current principal and resource ownership must be checked again. A session identifier is not an identity, and a conversation identifier is not a delegation grant.
Middleware and Hook Gaps
Teams often place policy in model-request hooks: inspect prompts, limit tools exposed to the model, require approval when the model emits a risky call, or redact data before inference. Those controls matter. They do not cover a call dispatched from restored history if the model is not invoked again.
Any path that can reach dispatch without traversing the hook must still encounter equivalent enforcement. The safest architecture does not depend on every route reproducing model-layer policy; it places non-bypassable authorization at the dispatcher and handler.
Retry and Recovery Ambiguity
After a timeout, the orchestrator may not know whether a tool executed. Blind retry can duplicate a payment, message, deployment, or deletion. Marking the call complete without proof can hide an incomplete operation. Histories therefore need explicit call state, idempotency support, and reconciliation rather than inferring completion from message position.
Trust Classes for Agent Messages
Not every message deserves the same treatment. A practical classification helps the runtime decide what can influence execution.
| Message or event source | Default trust | Permitted use | Must not do alone |
|---|---|---|---|
| Current user input | Untrusted content from authenticated principal | Inform model and policy within scope | Directly authorize a tool or another principal's resource |
| Current model output | Untrusted proposal | Propose tool and arguments | Grant capability, bypass approval, or select hidden credentials |
| Client-supplied history | Untrusted serialized context | Reconstruct visible conversation after validation | Create authoritative pending calls |
| Server-stored signed history | Integrity-protected record | Support display and carefully controlled resumption | Override current authorization or policy |
| Tool result | Untrusted data from a specific invocation | Satisfy the exact pending call after correlation | Act as instruction or approval for another call |
| Human approval | Trusted only within its authenticated transaction | Permit the exact reviewed operation | Approve changed arguments, targets, or future calls |
| Tool metadata/annotations | Descriptive and potentially untrusted | Help UI and model planning | Establish actual safety, read-only behavior, or privilege |
| Policy decision | Authoritative for its defined context and lifetime | Allow or deny the bound operation | Remain valid after principal, resource, or policy changes without review |
The classification makes one principle visible: model output, tool result, and history are all data until trusted code turns a specific proposal into an authorized dispatch.
Design a Canonical Run State Machine
Message order is a weak foundation for execution. Maintain server-side run state with explicit transitions:
AWAITING_USER
-> MODEL_PENDING
-> CALL_PROPOSED
-> APPROVAL_PENDING (when required)
-> DISPATCH_AUTHORIZED
-> EXECUTING
-> SUCCEEDED | FAILED | UNKNOWN
-> MODEL_PENDING | AWAITING_USER | TERMINATED
Each transition should be atomic and attributable. Only trusted orchestration code can move CALL_PROPOSED to DISPATCH_AUTHORIZED. A client cannot set EXECUTING by submitting an assistant message. A model cannot set APPROVAL_PENDING to approved. A retry worker cannot turn UNKNOWN into a new invocation without idempotency or reconciliation policy.
Separate Transcript from Control State
The transcript can remain a useful projection for models and users. Control state should live separately and reference the transcript through immutable IDs or digests. This separation allows the system to display imported history without treating it as runnable, redact content without changing pending-call state, and compact old messages without resurrecting operations.
For every call, retain fields such as:
- Run, turn, and call identifiers.
- Authenticated principal and tenant at proposal time.
- Tool registry identity and version.
- Canonical argument digest and resolved resource summary.
- Proposal source and model/request correlation.
- Required approval policy and approval record.
- Dispatch authorization decision and policy version.
- Idempotency key and execution attempt count.
- Outcome, effect reference, and reconciliation state.
- Cancellation, revocation, and expiry timestamps.
Avoid storing secrets or unrestricted model reasoning merely because auditability matters. Record the minimum structured facts needed to explain authority and effect.
Enforce Authorization at Dispatch and Inside the Tool
The dispatcher is the point where a proposal becomes an invocation. It should perform cross-cutting checks:
- Load the authoritative pending call from server-side state.
- Authenticate the current actor or workload and bind it to the run.
- Confirm the call is in a dispatchable state and not expired, canceled, or consumed.
- Resolve the exact tool implementation from a trusted registry.
- Validate arguments against a pinned schema and semantic policy.
- Resolve target resources and check tenant/object authorization.
- Verify required human or system approval against the argument digest.
- Apply environment, network, data, rate, cost, and destructive-action guardrails.
- Atomically reserve execution and idempotency state.
- Invoke the tool with least-privileged, scoped credentials.
The tool handler must also authorize resources at the protected effect. A central dispatcher may know that a user can call read_document, while the document service knows whether that user may read document 42. Defense at both layers prevents generic tool permission from becoming object-level access.
Approval Must Bind to What Will Execute
A prompt saying "Allow this tool?" is inadequate if arguments can change after approval. The approval record should bind at least the principal, tenant, tool identity, canonical arguments or their digest, resolved target summary, purpose, expiry, and run. If resolution changes between approval and execution, policy should require a new decision.
For high-impact actions, show the meaningful effect rather than raw JSON alone: recipients and data classification for messaging, environment and resource for deployment, path and operation for file tools, account and amount for financial actions. Human approval is effective only when the user can understand what is being authorized.
Tool Descriptions Are Not Security Policy
Agent protocols allow tools to publish names, descriptions, schemas, and annotations. These fields improve discovery and user experience. Unless they come from a fully trusted source and are verified against implementation policy, they must be treated as hints.
A malicious or compromised tool can describe itself as read-only while writing data. It can claim to be idempotent while creating duplicate effects. A tool description can contain prompt injection. A schema can omit semantic constraints enforced only by the downstream service.
Maintain an operator-controlled tool policy registry with:
- Trusted server and package identity.
- Tool version and schema digest.
- Allowed principals, tenants, and environments.
- Required scopes and downstream credential profile.
- Data classifications allowed in arguments and results.
- Network and filesystem boundaries.
- Destructive, financial, privacy, or external-communication flags.
- Approval and dual-control requirements.
- Rate, concurrency, cost, and timeout limits.
- Idempotency and compensation strategy.
Metadata can suggest defaults, but deployment policy must decide authority.
How to Test History as an Attack Surface
Conventional API scanners rarely understand agent run state. Testing needs stateful mutations and protected-effect oracles.
Establish a Safe Baseline
Use a synthetic user, tenant, run, and tool set. Tools should operate on disposable records or canary resources. Record a normal sequence from user request through model proposal, optional approval, dispatch, tool result, and continuation. Capture server-side state transitions, not private model reasoning.
Mutate Structure, Not Just Text
Test message and state variations such as:
- Remove the final user cancellation or clarification.
- Leave an assistant tool call unresolved.
- Add a tool result without a matching call.
- Duplicate a call or result ID.
- Reorder call and result events.
- Move a call between runs, users, or tenants.
- Replay a completed call after policy or membership changes.
- Change arguments after approval.
- Restore a history created under an older tool schema.
- Import a transcript containing assistant tool-call objects.
- Resume from a client-supplied history without invoking the model.
- Trigger retry while the original execution outcome is unknown.
These are protocol-state tests, not prompt variations. Each should have an explicit expected state transition.
Exercise Every Dispatch Path
Map model-originated dispatch, restored-run dispatch, direct API invocation, queued retries, scheduled jobs, sub-agent delegation, human replay, and administrative recovery. Confirm that all routes reach the same authorization and argument-binding controls. A secure interactive chat path does not prove a safe retry worker.
Verify Protected Effects
Do not use an error message as the sole oracle. Inspect the disposable file, database row, outbound-message sink, deployment canary, secret-access log, or other protected resource. A denied mutation must create no effect, no queued job, no reusable approval, and no authoritative pending call.
For teams using SecureHup, this form of validation can turn a subtle history-state hypothesis into a concrete answer about whether a protected action actually occurred. The value to users is faster feedback, less time spent debating speculative agent risks, a clear trace developers can remediate, and a replayable check that the dispatch boundary stays closed after code changes. That outcome-oriented use is more useful than describing platform internals or adding a promotional section.
History-to-Tool Integrity Test Cases
| Test family | Mutation | Expected enforcement | Evidence |
|---|---|---|---|
| Unresolved call | History ends with assistant tool request | No dispatch without authoritative current-run state | No protected effect; denied transition logged |
| Trailing-message removal | Sanitizer drops a later user event | Earlier call is invalidated or fresh decision required | State remains non-dispatchable |
| Cross-run splice | Move call object to another run | Run/call binding fails | No new attempt in either run |
| Cross-principal resume | Open run as another user | Principal/run and resource authorization re-evaluated | Foreign user cannot trigger effect |
| Result injection | Submit result for absent or different call | Correlation fails | Model/control state unchanged |
| Duplicate dispatch | Replay completed call | Idempotency or terminal state blocks effect | One effect reference only |
| Approval drift | Change target after human approval | Approval digest mismatch | New approval required |
| Policy drift | Revoke role after proposal | Current authorization denies dispatch | No stale capability use |
| Tool-schema drift | Resume under changed schema/version | Pinned version check or revalidation | No ambiguous argument execution |
| Hook bypass | Resume without model request | Dispatcher still applies policy | Same denial as interactive path |
| Unknown outcome | Retry timed-out non-idempotent tool | Reconcile or block automatic retry | No duplicate effect |
| Metadata deception | Tool claims read-only but test canary detects write | Operator policy and sandbox contain effect | Mismatch recorded and tool disabled |
The matrix should include allowed controls as well. A valid current-run call with correct authorization and approval must execute once and produce an attributable result. Tests that only expect denial can hide a broken system behind universal failure.
Evidence for Tool-Call Integrity
A high-quality finding should answer:
- Where did the history or call originate?
- Which fields were attacker-controlled or mutable?
- What transformation, restoration, or resume path exposed the call?
- Was the model invoked in the exploit sequence?
- Which hooks or controls were skipped, and which still ran?
- Which principal, tenant, tool, and resource were involved?
- Was approval absent, stale, or bound to different arguments?
- What protected side effect occurred?
- Was execution duplicated or merely proposed?
- What is the minimal safe replay sequence?
- Which policy or code change blocks it?
- Does the fixed build deny the sequence while preserving the legitimate baseline?
This structure prevents overstatement. Exposing an unresolved call to a model is not the same as dispatching it. Dispatching a no-op canary is not the same as accessing production data. Bypassing a model hook can be serious, but the final impact depends on dispatcher and handler controls. Evidence should preserve those layers.
Remediation Patterns
Reject Authoritative Calls from Client History
Treat client-supplied assistant tool-call objects as display or model context only. If the system supports trusted resume, use a server-issued run handle to load authoritative state rather than reconstructing it from submitted messages.
Make Sanitization Semantics-Preserving
Define valid conversation grammar and explicit repair outcomes. When a transformation would expose or alter a pending executable event, invalidate the turn or require a fresh decision. Test sanitizers with security-state assertions, not just "API no longer errors."
Bind Calls and Results to Runs
Use server-side call records with run, turn, principal, tenant, tool, argument digest, and status. Accept results only for a currently executing call. Never infer authority solely from a call ID carried in a message.
Reauthorize at Dispatch
Evaluate current principal, membership, resource ownership, tool policy, environment, approval, and limits immediately before execution. Use short-lived delegated credentials scoped to the concrete operation rather than global server secrets where possible.
Make Effects Idempotent or Reconciled
Give protected operations idempotency keys tied to authoritative call IDs. For operations that cannot be safely retried, represent UNKNOWN explicitly and require reconciliation or human review. Do not turn network uncertainty into duplicate authority.
Keep a Kill Switch and Revocation Path
Operators need to disable a tool, server, credential profile, tenant, or run quickly. Pending approvals and calls should become invalid when underlying authority is revoked. Test the revocation path under in-flight and queued conditions.
Release Controls for Historical Tool Calls
History-state regressions often arrive through framework upgrades, message serialization changes, compaction logic, new protocol support, retry refactors, or additional tool adapters. Connect those diffs to focused replay suites.
Maintain tests at several levels:
- Parser and grammar tests for message sequences and unknown fields.
- Property tests for call/result pairing, uniqueness, and terminal states.
- Sanitizer tests that prove control-state preservation.
- Authorization tests across principal, tenant, resource, tool, and policy changes.
- Approval tests covering argument and target mutations.
- Idempotency tests with concurrent and uncertain outcomes.
- Adapter conformance tests proving every dispatch path reaches shared enforcement.
- End-to-end canary tests verifying protected state in a sandbox.
- Upgrade tests for stored histories created under previous schema and tool versions.
Operational metrics should favor control quality: unauthorized dispatch attempts blocked, calls without authoritative run records, approval-digest mismatches, cross-principal resume denials, duplicate effects, unknown outcomes awaiting reconciliation, stale pending calls, tool-policy version drift, and mean time to revoke a risky tool. Raw count of model-generated tool calls is not a security metric.
A Review Checklist for Agent and MCP Tools
Before enabling a tool-capable agent in a multi-user environment, verify:
- Client-submitted history cannot create authoritative pending calls.
- Transcript content and execution control state are stored separately.
- Every call and result is bound to run, turn, principal, tenant, and tool identity.
- Sanitization cannot expose an earlier pending call by dropping later context.
- Restored runs re-evaluate current authorization and policy.
- Every dispatch path reaches the same central controls.
- Tool handlers enforce object-level and tenant-level authorization near the effect.
- Human approvals bind exact arguments, targets, purpose, and expiry.
- Tool metadata and annotations are treated as hints unless independently trusted.
- Sensitive tools use least-privileged credentials and constrained environments.
- Idempotency or reconciliation covers retries and unknown outcomes.
- Audit records distinguish proposal, approval, dispatch, execution, result, and effect.
- Operators can revoke pending calls, approvals, tools, and credential profiles.
- Synthetic history mutations and canary effects run in regression testing.
Frequently Asked Questions
Can an agent conversation history really execute a tool?
The text itself does not execute. Risk appears when an orchestration runtime interprets structured history as authoritative pending state and dispatches a tool call. Whether that happens depends on the framework and application architecture, which is why testing must verify the actual dispatch and protected effect rather than assume it.
Is sanitizing malformed history enough to prevent tool-call injection?
No. Sanitization can reduce ambiguous sequences and is useful defense-in-depth, but it may be bypassed, may alter semantics, or may not run on every path. Current authorization, approval, scope, and resource checks are still required at dispatch and inside sensitive tools.
Should tool calls from model output be trusted more than client history?
They have different provenance, but model output is still a proposal, not authority. A current model-generated call can be influenced by untrusted content and must pass policy. Client history is even less suitable as execution state because it may be altered or replayed. Neither should independently grant capability.
How should human approval be recorded?
Bind approval to the authenticated approver, principal on whose behalf the action runs, tenant, exact tool version, canonical arguments or digest, resolved targets, purpose, expiry, and run. If any security-relevant field changes, obtain a new approval.
What is the safest way to test tool-call integrity?
Use synthetic users and tenants, disposable or canary resources, scoped credentials, and a sandbox. Mutate history and state, exercise every dispatch path, and inspect protected effects. Avoid production secrets, destructive targets, real recipients, and uncontrolled third-party systems.
Current Authority Is the Real Dispatch Boundary
Once transcript content is separated from server-authored call and approval records, the underlying design question becomes much easier to see. A tool call is valid only while its run, principal, tenant, arguments, target, policy, approval, and idempotency state still agree. An upstream flag can contribute to that decision, but it cannot preserve authority after the surrounding context has changed.
We would begin by tracing every route into the dispatcher and replaying a small set of stale or cross-principal histories against harmless canaries. A well-formed transcript is not the success condition. What matters is that historical context cannot produce a protected effect that the current execution context would refuse.
#AISecurity #AgentSecurity #MCP #ToolCalling #Authorization #ApplicationSecurity #AutonomousPentesting
Find issues like this in your own codebase
SecureHup runs this exact exploit chain — discovery, enumeration and cross-tenant validation — on every deployment, automatically.