Agent safety usually means a gate: something between the model and the world that decides whether an action is allowed. That design has a hole in it. When the model emits a call to a tool no server advertises, or passes a field no schema declares, there is no real action for the gate to rule on — so it waves it through. Probing ten hosted models produced 322 such calls, and a check that simply asks “is this in the registry?” before the gate stopped every one, while rejecting none of the well-formed ones.
A policy gate answers a question about a real capability: may this agent, in this state, send that email? The calls in this paper never get that far. The model names a tool that is not in the registry, or invents an argument the schema does not declare, or passes a string where a number is required — and a gate that only knows how to allow or deny real tools has nothing to decide.
Step through the classes. Each one shows the call the model emitted, then what happens under a policy gate alone and under a registry check placed in front of it.
Four of the five are settled by information you already have: the registry of tools and their signatures. The fifth — a real tool called with another tool's argument shape — is only catchable when the borrowed shape happens to violate the named tool's signature. When it does not, the call is schema-valid and indistinguishable from a correct one, which is a tool-selection problem rather than a validation problem.
The same ten models were probed on two surfaces. On one, the decoder is constrained to the supplied schema, so a fabricated tool name largely cannot be produced in the first place. On the other — tools described in the prompt, the model emitting free-form JSON — nothing stops it, and the call arrives at your executor looking like any other.
Flip the surface.
Size does not save you. On the free-form surface an 8B model fabricated at 45% and the 675B model from the same family at 55%; the two sizes of another open-weight family sat at 55% and 57%. The frontier model in the set was the best behaved at 13%, which is still one call in eight. This is a property of the interface you expose, not of the model you bought.
The proposed resolver is deliberately dull, needs no training and no model call, and runs ahead of the policy gate rather than beside it. It asks three questions in order, and a call that fails any of them is rejected outright.
Step through the tests and watch which classes stop being executable.
Against the 322 hallucinated calls found in the wild, the gate-only stack executed all of them and the full stack executed none — and no well-formed call was rejected in the process. The residue is the undetectable borrowed signature: 61 of 400 scripted attempts survive, because a schema-valid call to the wrong tool is not a validation failure. That number is the honest boundary of what a closed-world check can do for you.
The interesting part for anyone wiring up tool servers: when several servers are merged into one namespace, the hallucinations change shape. A name can be advertised by two servers at the same trust level, so the call is ambiguous rather than wrong. Or a low-trust server can advertise a name a trusted server owns, and a host that routes by name alone hands the call to the wrong provider — the model did nothing unusual at all.
Toggle between a single registry and a merged namespace.
The number worth remembering: the frontier model that emitted zero fabricated calls against a single schema-constrained registry hallucinated on 57% of its probes against the merged surface. Nothing about the model changed. Collisions and shadowed names are properties of the merge, so they are your host's problem — resolve names against a per-server registry with a trust order, and reject rather than guess when two servers claim the same name.
Put your own traffic through the paper's measured rates. The point is not the exact count — your prompts and tools are not theirs — but the shape: on a free-form surface the fabricated calls are a steady fraction of everything you run, and the only thing standing between them and your executor is whether something checked the name.
Three checks, in this order, before your gate: is the name in the registry, are the keys exactly the declared ones, do the values match their declared types and enums. None of it needs a model, and the residue it cannot reach — a valid call to the wrong tool — is at least a problem you can name and measure separately.