/p/2026-09-19 · explainer
Paper explainer · 2609.19425 · Ganesh Iyer

It called a tool that does not exist. Nothing was asked.

Agent safety usually means a gate: something between the model and the world that decides whether an action is allowed. That design has a hole in it. When the model emits a call to a tool no server advertises, or passes a field no schema declares, there is no real action for the gate to rule on — so it waves it through. Probing ten hosted models produced 322 such calls, and a check that simply asks “is this in the registry?” before the gate stopped every one, while rejecting none of the well-formed ones.

01 · The failure

Five ways a tool call can be wrong, and only one of them is what gates were built for

A policy gate answers a question about a real capability: may this agent, in this state, send that email? The calls in this paper never get that far. The model names a tool that is not in the registry, or invents an argument the schema does not declare, or passes a string where a number is required — and a gate that only knows how to allow or deny real tools has nothing to decide.

Step through the classes. Each one shows the call the model emitted, then what happens under a policy gate alone and under a registry check placed in front of it.

Interactive · the five hallucination classes

Four of the five are settled by information you already have: the registry of tools and their signatures. The fifth — a real tool called with another tool's argument shape — is only catchable when the borrowed shape happens to violate the named tool's signature. When it does not, the call is schema-valid and indistinguishable from a correct one, which is a tool-selection problem rather than a validation problem.

02 · The surface

How you let the model emit the call decides how often it invents one

The same ten models were probed on two surfaces. On one, the decoder is constrained to the supplied schema, so a fabricated tool name largely cannot be produced in the first place. On the other — tools described in the prompt, the model emitting free-form JSON — nothing stops it, and the call arrives at your executor looking like any other.

Flip the surface.

Interactive · constrained decoding versus free-form JSON
0%30%60%
what the surface itself prevents

Size does not save you. On the free-form surface an 8B model fabricated at 45% and the 675B model from the same family at 55%; the two sizes of another open-weight family sat at 55% and 57%. The frontier model in the set was the best behaved at 13%, which is still one call in eight. This is a property of the interface you expose, not of the model you bought.

03 · The check

Three questions, asked before anything is allowed to be decided

The proposed resolver is deliberately dull, needs no training and no model call, and runs ahead of the policy gate rather than beside it. It asks three questions in order, and a call that fails any of them is rejected outright.

Step through the tests and watch which classes stop being executable.

Interactive · adding one test at a time
0200 trials400 trials

Against the 322 hallucinated calls found in the wild, the gate-only stack executed all of them and the full stack executed none — and no well-formed call was rejected in the process. The residue is the undetectable borrowed signature: 61 of 400 scripted attempts survive, because a schema-valid call to the wrong tool is not a validation failure. That number is the honest boundary of what a closed-world check can do for you.

04 · The merge

Connect a second tool server and two new failures appear that no single registry can see

The interesting part for anyone wiring up tool servers: when several servers are merged into one namespace, the hallucinations change shape. A name can be advertised by two servers at the same trust level, so the call is ambiguous rather than wrong. Or a low-trust server can advertise a name a trusted server owns, and a host that routes by name alone hands the call to the wrong provider — the model did nothing unusual at all.

Toggle between a single registry and a merged namespace.

Interactive · one server versus several, live surfaces
hallucinated calls that a naive host executed
stopped once the resolver runs first

The number worth remembering: the frontier model that emitted zero fabricated calls against a single schema-constrained registry hallucinated on 57% of its probes against the merged surface. Nothing about the model changed. Collisions and shadowed names are properties of the merge, so they are your host's problem — resolve names against a per-server registry with a trust order, and reject rather than guess when two servers claim the same name.

05 · On your agent

The cheapest validator you will ever ship illustrative

Put your own traffic through the paper's measured rates. The point is not the exact count — your prompts and tools are not theirs — but the shape: on a free-form surface the fabricated calls are a steady fraction of everything you run, and the only thing standing between them and your executor is whether something checked the name.

Interactive · your tool traffic, the paper's rates illustrative
malformed or fabricated calls reaching your executor
rejected by a registry check that costs a dictionary lookup

Three checks, in this order, before your gate: is the name in the registry, are the keys exactly the declared ones, do the values match their declared types and enums. None of it needs a model, and the residue it cannot reach — a valid call to the wrong tool — is at least a problem you can name and measure separately.

Results

What the paper actually measured

What it does not show

In practice