/p/2026-10-19 · explainer
Paper explainer · 2610.21128 · Zhang, Gondara and Novikova

They all agreed. That is not the same as being right.

Label-free evaluation runs on a proxy: ask the same question several ways and treat agreement as evidence of correctness. Across 23,170 perturbed financial questions and two frontier models the proxy is real — an answer below the agreement threshold is about eight times more likely to be wrong — and it still waves 60.23% of all wrong answers through as unanimous. In 77.7% of those cases every rephrasing returned the identical wrong answer, so there was nothing to disagree about. Raising the threshold flattens out at 0.6, and asking a second model family produces three and a half times more shared errors than independence would predict.

01 · The problem

certified — agreement ≥ 0.80
flagged — agreement below 0.80

02 · The mechanism

which model is answering

03 · The evidence

which question set

04 · Why I care

05 · Apply it

Your gate, your residual illustrative

Results

What the paper actually measured

What it does not show

In practice