A wrong action does not always fail loudly. It can land somewhere plausible and wrong, return no error, and leave the agent to carry on as if it had worked. This paper pins each action's intended effect down before any executor runs, which makes silent failure something you can count rather than something you infer from the wreckage — and the count is brutal in one specific place. Apply a code edit by line number, shift the file by a single line, and 99.1% of files are corrupted with no signal at all. The same edits expressed as matched content fail at 0%: they refuse, loudly, and a refusal is recoverable.
The framing the paper insists on is that an action has three possible endings, not two. It can do what was intended. It can refuse — the pattern did not match, the file was locked, the command would not parse — and the agent sees the refusal and tries again. Or it can take effect somewhere other than intended and report nothing, which is the only one of the three that leaves the system wrong and the agent confident.
Most tooling treats the second and third as the same event, because both are "the edit did not do what I wanted". They are not the same event at all. Step through them.
Getting the third category countable is the paper's actual method. Rather than watch an agent work and try to infer afterwards whether an edit went astray, the authors fix each action's correct effect by construction — they synthesise the target state first, then run the apply step in isolation and compare. Silent failure stops being a story about a debugging session and becomes a rate you can put a confidence interval on.
640 edits over 224 Python files, each applied six ways, under two perturbations that mimic what actually happens in a live repository: the file gets reindented, or something inserts a line above the target. Every format is perfect when nothing has moved. Move the file by one line and they split into two populations with nothing in between.
Pick a format and shift the file underneath it.
The dividing line is not fuzziness or strictness — it is what the edit points at. Content-anchored formats name the text they mean to replace, so when the text is not there they cannot pretend: search/replace and unified diff sit at exactly 0% silent failure in every condition tested. Location-anchored formats name a position, and a position is always valid. Line 42 exists whether or not it still holds what you meant, so the edit lands and the tool reports success.
Editing a whole function by name looks like the safe middle and is not: on files nobody perturbed at all, it lands on the wrong function 12.7% of the time — duplicate names across classes, nested definitions, decorated wrappers. And aggressive fuzzy matching, the usual fix for brittle exact matching, buys its 97.7% apply rate by starting to guess: 2.3% silent on single-hunk edits, rising to 39.3% once an edit has several hunks.
The same argument runs on the agent's other action channel. Before a command executes, three deterministic tests can be run against the local machine: does it parse, does the binary exist, and is every flag one the binary actually accepts. Nothing here involves a model. Add the layers one at a time over 9,930 commands spanning 482 installed tools.
The structure of that result is the useful part. The first two checks are oracle-exact: a command that will not parse will not parse, and a binary that is not installed is not installed, so they raise zero false alarms while catching half of everything. Only the flag check can be wrong, because it depends on scraping help text, and every one of the 198 false positives comes from it. So the two halves want different treatment, which is the next section.
A check that is sometimes wrong should not have one setting. The paper's gate blocks what its exact checks are certain about, warns about what its coverage-bounded check merely suspects, and abstains entirely on the one token class it knows it cannot read — single-dash multi-character flags, where a tool might mean one letter or a word. Abstaining there is free by construction, because no injected error used that form.
Three points of false-positive rate for nothing — same recall, fewer good commands stopped — simply by declining to have an opinion where it has no evidence. The same idea, applied to the edit channel, is what turns the 99.1% number around. An applier that finds the target by content, requires at least two lines of anchor, demands a similarity of 0.90 and a clear margin over the runner-up, and refuses when any of that is missing, logged one silent misapplication in 8,320 trials.
That is the trade, stated honestly: on perturbed single-hunk edits the careful applier will attempt about half of what the aggressive one attempts. The half it declines come back to the agent as clean refusals it can act on, rather than as files that are quietly wrong. And the thresholds are a dial rather than a cliff — swept across a similarity floor of 0.85 to 0.95 and a margin of 0.10 to 0.30, the silent rate stays under 0.05% while the share of edits attempted moves smoothly.
Set how many edits your agents apply in a day and how often a file moves under them between read and write — a concurrent edit, a formatter, another agent in the same repository. The per-edit rates are the paper's measurements; multiplying them by your traffic is arithmetic, and your repository is not the paper's 224 files.
The asymmetry is what makes this worth an afternoon. A clean refusal costs one extra turn. A silent corruption costs a file that looks fine, passes review because the diff was never shown to anyone, and surfaces days later as a bug nobody can trace to an agent. Those two are not the same failure with different severities — they are different kinds of thing, and the format you pick decides which one you get.