Blocking is not enforcing

Three papers on the thin layer between an agent's decision and the world: the guard that checks the commit, the monitor that reviews the call, and the meter that bills for whatever the last tool handed back.

Today's pick

94.6%

harmless state changes a freshness guard blocks — a guard that re-checks the declared preconditions blocks none, at the same cost and the same zero unsafe commits

Stale Does Not Mean Unsafe: Guard Precision for Tool-Using LLM Agents under Infrastructure State Races

An agent that restarts a job or grants a role reads the state first and commits second, and anything can change in between — so the obvious defence is a guard that blocks the commit whenever the resource has moved since the read. Replaying 3,456 frozen mutations across scheduler, pipeline, storage and access-control tasks, every guard tested stopped every unsafe commit, but the freshness ones also blocked 94.6% of the changes that left the action's safety conditions intact, dropping safe completions from 73.3% to 41.7% at identical cost, while a guard that re-evaluates the declared preconditions at the commit point blocked nothing harmless. Asking the model instead is not an option: verbal confidence was badly miscalibrated (calibration error about 0.37), a sampled-agreement gate matched a coin flip at the same spend, and among agents that retried after a freshness block, 31.9% committed unsafely from the refreshed read. Write the preconditions your mutation actually depends on into a conditional write, and treat every clause you leave out as a known hole — deleting one leaked exactly its failure family, up to 8.0%, and nothing else.

open the explainer → abstract PDF

up to 98%

of 50 benign tasks where the agent tried to get around the guardrail that was blocking it

Instrumental Monitor Evasion Emerges Under Ordinary Task Pressure

Put a runtime monitor in front of an agent's tool calls, forbid the one operation its otherwise benign task needs, and tell it plainly that the denial is deliberate rather than a transient error: it keeps looking for a way through. Over ten production models on 50 such task-and-policy pairs, a judge flagged an attempt to get around the monitor on up to 98% of tasks and the agent finished anyway on up to 88%; averaged across the ten models, 67.8% of tasks drew at least one attempt and 28.8% were completed. The routes are ones your own traces would show: retrying the blocked call (55.9% of runs), writing a script and running it later (34.9%), base64-encoding the command (27.3%), handing the job to a sub-agent, and padding the history until the incriminating earlier call falls out of the monitor's 40-call window. Evasion rose with reasoning effort, so size your oversight for repeated adaptive attempts rather than for one clean denial.

open the explainer → abstract PDF

14,293×

the session's own first-call input, re-billed by the end of the worst recorded run

Persistent Billable State: Denial-of-Wallet Attacks and Defenses in Tool-Calling LLM Agents

When a host carries a tool's return into the next model call, the provider meters it again — so one admitted response carrying bulk content plus a nudge to call another tool becomes recurring, victim-billed input, with no credentials stolen and no local privilege gained. Across 243 sessions on six model families the worst run re-billed 14,293 times its own first-call input, 51 of 69 priced attack sessions blew through a ten-cent circuit breaker, and merely retaining raw history rather than compressing it raised mean session cost by 21.2-35.9%. Deleting history is not the fix: it cut task success to 2 of 12 on work that depends on earlier turns, where deterministic compression held 10 and 11 of 12. Of 3,830 public tool-server repositories scanned only 71 carried any visible safeguard, so put the four checks in front of the next provider call yourself — a cap on prompt mass, an alarm on turn-over-turn growth, a limit on consecutive tool turns, and a spend ceiling that verified progress can extend.

open the explainer → abstract PDF