You cannot see whether an answer is right, but you can watch the model become sure. Resample the answer from every prefix of a chain of thought and you get a trajectory: how undecided the model still was at each step. This reproduction, registered before the runs, splits that trajectory into two readings and finds only one of them carries signal. Whether uncertainty fell at every step is worth +9.6 points of accuracy on grade-school word problems and +27.5 on competition maths. How far it fell in total is worth −0.02 correlation on one benchmark and +0.41 on the other — which is to say, nothing you can build on.
Entropy here is not a property of the model's token distribution. It is measured over answers. Take the chain of thought so far, cut it at step k, and ask the model to finish from there five times at temperature 0.7 with a 150-token budget. Pull the final answer out of each continuation and count how many distinct answers you got. That count — strictly, the Shannon entropy of that answer distribution in nats — is how undecided the model still was at step k.
Change how the five continuations split and watch the number move.
Two things follow from the definition. First, this is expensive: every step you instrument costs five extra completions, which is why section 05 matters. Second, it is model-agnostic — no logits, no weights, no access to anything but the sampling API you already call.
Run that measurement at every step and you get a trajectory. The original 2026 result, which this paper set out to reproduce, claims a dissociation between two readings of it. The shape is a yes-or-no question: did the entropy fall at every step, within a tolerance of 0.01 nats? The magnitude is a number: how much lower was the last step than the first?
Here is why they are not the same thing. Drag the middle step. The first and last values never move, so the total drop is pinned — but the answer to the shape question flips.
A chain that gets more confused halfway through and recovers has the same total drop as one that settled steadily. The claim under test is that the first kind is measurably less likely to be right — that a model which changed its mind mid-chain has told you something, even though it arrived at the same confidence in the end.
This is a reproduction with its hypotheses and success criteria frozen at OSF before the confirmatory runs, which is the reason to take it seriously: the analysis could not be shopped after the fact. Four open-weight models — Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3, Llama-3.1-8B-Instruct and a reasoning-distilled DeepSeek-R1-Distill-Qwen-7B — ran on grade-school word problems and competition maths, with every interval a 95% percentile bootstrap over 1,000 resamples.
Switch benchmarks and watch both signals at once.
Read the magnitude column carefully, because it is the more interesting failure. On the grade-school set its correlation with correctness is −0.018, inside the preregistered band for “no effect”. On competition maths it is +0.414, comfortably outside it. A signal that is null on one benchmark and strong on another is not a null result and not a positive one — it is a signal whose meaning depends on the task, which is exactly what you cannot ship a threshold against.
The most useful finding is a side result. Forget the whole trajectory and keep only the entropy left at the final step. Ranking chains by that one number discriminated correct from incorrect better than the all-steps monotonicity flag by area under the ROC curve in all eight model-and-benchmark cells, and by risk-coverage area in six to seven of them.
The practical difference is not the accuracy, it is the shape of the control. A flag gives you one operating point: escalate every non-monotone chain, whatever share of traffic that turns out to be. A score gives you a dial.
The distributions above are a drawing, not the paper's data — it reports rank-ordering results, not densities. What is the paper's is the ordering claim itself, and that is the part you can act on: if you are going to spend a second model call on some fraction of answers, spend it on the highest final-step entropy rather than on a monotonicity test.
Now the arithmetic that decides whether any of this ships. Every instrumented step costs m extra completions of up to 150 tokens each. The paper used m = 5 and capped chains at eight steps, so a fully instrumented chain costs up to 40 extra completions per answer. Final-step-only costs m.
That gap is the whole engineering argument. At eight steps, instrumenting everything costs more than simply answering the question eight times and taking a majority vote — at which point you should do that instead. Final-step-only lands in the range where a reliability signal is worth its own budget: a handful of short completions, spent on the answers most likely to be wrong, feeding a router that sends those to a stronger model, a tool, or a person.