/p/2026-10-27 · explainer
Paper explainer · 2610.32110 · Jeffries

The tool was right. The sentence was not.

A search tool fetched at most 20 rows, so its formatter printed “Found 20 document(s)” — while 4,772 documents actually matched. Every word of that line is true; the count it omits is the one the user asked for. The agent answered that the archive contains exactly 20, and the verifier checking that answer against the tool output accepted it 90 of 90 times, because the verifier reads the same sentence and cannot see the missing total either. Add one line stating the cap and the true count and the same false answer is rejected 90 of 90 times. This is a string-formatting bug class, not a model problem.

01 · One line, three consequences

02 · Four families, one flat line

step through the four defect families — the panel shows both wordings

03 · The verifier reads the same string

04 · Ask it the other question

05 · Your own tool layer illustrative

Results

What the paper actually measured

What it does not show

In practice