/p/2026-10-08 · explainer
Paper explainer · 2610.07354 · Pishehvar, Morandi and Viswanathan

The router was reading the question.

Before paying for the big model you want a cheap signal that says whether escalating this query will help. Semantic entropy is the obvious candidate, and on grade-school maths with a pair twelve times apart in size it earns its place: a ranking score of 87.1 against 64.9 for a guess that never calls a model, and nine points of accuracy over escalating at random for the same spend. On a synthetic arithmetic ladder the same signal’s best-looking number was nothing at all — a regex reading only the question matched it exactly, 83.9 against 83.0 — and at ten samples a query the router cost 39% more than simply always calling the large model. The paper’s real output is five checks that catch all of this before it gets reported as real.

01 · The problem

Beating a coin flip is not the same as adding information

pick a benchmark

and how many samples the signal pays for

02 · The mechanism

Which definition of “escalation worked” are you measuring?

pick a definition of the thing being predicted
now walk the difficulty bands under that definition

03 · The method

Five ways to fool yourself, and what each one caught

tap a check

      

04 · The bill

At ten samples a query, the router is the expensive option

05 · In your own product

Whether a router can win on your numbers illustrative

precondition one — is there any room
precondition two — can it be the cheap option

Results

What the paper actually measured

What it does not show

In practice