Reading a model’s written reasoning to catch a bad answer only works if the writing is driving the answer. Corrupt one step of a correct chain, cut the chain off there and force the model to continue: on grade-school arithmetic it ignores the damage and answers correctly anyway 94.5% of the time, while on the hardest multistep arithmetic it follows the bad step to a wrong answer in 64.5% of runs. Splitting the variance over 28,584 continuations puts 98.8% of the explained variation on task difficulty and 0.8% on how the step was corrupted.
The case for reading a model’s chain of thought in production is that the trace shows you the answer being built, so a bad answer should be visible as a bad step. That argument has a hidden premise: the written steps have to be what produced the answer. If the model would have said the same thing with a different trace, the trace is a report rather than a cause, and monitoring it tells you nothing about the answer.
The paper tests the premise directly. Take a chain the model got right, damage one intermediate step, truncate everything after it, and force the model to continue from the damaged prefix. Three things can happen: it silently ignores the damage and lands on the right answer anyway, it notices and repairs the step, or it follows the bad step to a wrong answer. Only the third is a trace that was carrying the answer.
Step across the three task families and watch which of the three it does.
On grade-school arithmetic the model ignored the corruption 94.5% of the time. On the hard reasoning benchmark it followed the corruption 40.9% of the time. Same model, same intervention, opposite conclusion about whether the trace means anything.
The intervention is deliberately blunt, and its bluntness is the point. Work through it.
This measures something narrower than whether a trace is faithful to the model’s internal computation. It measures whether the written text constrains the answer: if the model continues from a damaged prefix as if the damage were not there, the prefix was not constraining anything. The paper calls that behavioural rather than mechanistic, and the distinction matters because behavioural is the property a monitor depends on.
It would be convenient if this were a story about corruption types — some damage is easy to spot, some is not — because then a monitor could be tuned for the subtle kinds. The matched two-by-two rules that out. Move one axis at a time and see which one moves the number.
The variance partition over all 28,584 continuations is blunt about it: 98.8% of the explained variation is task difficulty, 0.8% is perturbation type, and 0.4% the interaction. The same gradient reappears inside a single benchmark at a fixed answer format — high-school psychology questions propagate corrupted steps 7.5% of the time, obscure world facts 53.7%, a sevenfold spread with nothing changing but how hard the model finds the subject.
There is no perturbation-shaped knob to tune, and no format-shaped one either. The only thing that predicts whether a written step is carrying the answer is how hard the model finds the question.
Plot every model-and-task pair against the model’s base accuracy on that task and the relationship is visible without a statistic: the better a model is at a task, the less its written steps constrain its answers. That holds within a model across tasks, and it is why difficulty has to be defined model-relative rather than by a human’s sense of which benchmark is hard.
The reasoning-trained model breaks the pattern in the useful direction. It suppresses error propagation broadly — 2.7% on the mixed-knowledge benchmark against the instruction-tuned model’s 22.3%, and 16.8% against 40.9% on the hard one — and it does so by repairing damaged steps rather than ignoring them: self-correction runs at 46.0% and 42.3% where the instruction-tuned model manages 36.3% and 21.7%.
The last hope for a monitoring story is that the behaviour is at least detectable and steerable from inside. Half of that is true.
Linear probes on hidden states separate the three behaviours well above chance. Steering is where it fails: additive activation steering flips at best about a quarter of error-propagation cases, and none of 726 bypass cases. Readable, not controllable.
Suppose you read chains of thought in production — a cheap model flags traces whose steps look wrong, and you review or block what it flags. The value of that depends almost entirely on the mix of easy and hard requests you serve, in a direction that is the opposite of comforting.
On the easy share the monitor is reading text that is not driving the answer: it will flag bad-looking steps that produced perfectly good answers, and it will miss nothing, because there was nothing to catch. On the hard share the steps are real, so a monitor can genuinely see the error being formed — but that is also where the model is least likely to repair it on its own, so by the time a step is visibly wrong the answer downstream of it is already wrong.
That does not make trace monitoring worthless; it makes it a tool with a known coverage profile.