How to prevent double execution in asynchronous agent workflows
A policy match is not a one-use guarantee. Safe asynchronous execution needs a state machine that remains correct under retries, races, timeouts, and downstream failure.
Backend, platform, and reliability engineers
Key takeaways
- Reserve one execution atomically before the platform calls an external system.
- Use idempotency to return the same durable authorization across retries.
- Reconcile uncertain downstream outcomes before releasing a single-use mandate.
Why a stateless policy check fails
Imagine two workers find eligible inventory at nearly the same time. Both submit an action under the same single-use mandate. If each request only compares fields with the policy, both can receive a positive answer before either execution is recorded.
That race is not fixed by adding a “used” flag after the external call. The external action may already have happened twice. The authorization runtime must atomically secure the right to execute before either worker crosses the downstream boundary.
Use an explicit reservation lifecycle
When an eligible proposal arrives, the runtime creates an authorization and reserves the mandate in the same database transaction. Only that authorization receives an executable state. Competing proposals receive a non-allow result even if their fields would otherwise match.
- Reserved: one authorization owns a temporary right to execute the exact proposed action.
- Committed: the platform confirms external success and the mandate becomes consumed.
- Cancelled: the platform confirms it will not execute, allowing release only when policy permits.
- Expired: the execution window closes automatically and no late execution is valid.
- Revoked: the grantor or platform terminates unused authority according to the contract.
Make retries return identity, not another decision
Networks time out, workers restart, and clients retry. An idempotency key should bind one organization, operation, mandate, and request payload to one durable authorization identity. Replaying the same request returns the original authorization in its current state.
Reusing the key with a different payload must be rejected. Creating a fresh authorization for an ambiguous retry can turn a harmless transport failure into duplicate authority.
Treat external execution as an uncertain boundary
The authorization service cannot assume an external booking or payment succeeded merely because the request returned. A timeout may mean failure, success, or an unknown outcome. The platform needs an external idempotency key or reference and a reconciliation path before it decides whether to commit or cancel.
Cancellation should not be a generic escape hatch. If an external action may already have succeeded, releasing the mandate could permit a second execution. The platform should cancel only after it can establish that the external system will not complete the action.
Test the races deliberately
Unit tests around comparison logic are necessary but insufficient. Exercise the real database transaction and the complete HTTP lifecycle with overlapping requests.
- Two eligible proposals start concurrently; exactly one becomes executable.
- The winning request times out and retries with the same idempotency key.
- Commit and cancel arrive concurrently for the same authorization.
- Revocation races with authorization creation.
- The execution deadline passes immediately before commit.
- A downstream timeout is reconciled without releasing authority prematurely.
“ALLOW” is safe only when it represents both a successful comparison and an exclusive, time-bounded reservation.