The compressor kept the tokens and lost the patch

Three papers about what the model is holding when it answers: the context assembled for a coding agent, the reasoning trace that outvotes evidence retrieved a moment later, and the critic that only earns its keep once the agent says it is finished.

Today's pick

46.7% vs 40.3%

issues resolved at the same token budget by picking whole files and showing them verbatim, against the best of three text compressors

Skip the Compressor: Coding Agents Fix More Issues When They Pick Whole Files

When the repository will not fit in the context window the reflex is to compress the text or retrieve the most relevant chunks, and at a fixed budget over 300 recent issues both lost to simply choosing whole files and showing them unaltered: 43.7% and 46.7% resolved on two models, 6.3 points above the best of three compressors and 9.3 to 12.7 above the best of three retrievers. Both failure modes are mechanical. Chunk retrievers rank whatever echoes the issue's wording, so they spend the budget on tests and documentation and leave 26% to 29% of issues with no file that needed editing anywhere in context, against 3.4% for file selection; compressors keep the right files but rewrite the code inside them, so the patch no longer matches the file on disk, and patches applied 98.0% of the time from verbatim files against 70.7% from the most aggressive compressor. Deleting a fifth of the tokens from otherwise good contexts cost 9.7 points on its own: spend the budget deciding which files to show, and keep the bytes intact.

open the explainer → abstract PDF

+29.9 to +60.6 pts

more often the answer followed the current evidence when it came from a separate call instead of a continuation of the model's own reasoning

When Reasoning Traces Conflict with Evidence: How Answer Configuration Shapes Evidence Following

If a model reasons first and fresher evidence lands before it answers, the shape of the answer call decides which source wins: continuing inside the model's own thinking block repeated the now-stale conclusion in 65.6% to 87.0% of conflicts, while a separate answer call with the new evidence placed last followed that evidence 29.9 to 60.6 points more often, across three model families on two multi-hop question sets. Accuracy when trace and evidence agree barely moves either way, within 4.7 points, so a trace being useful and a trace being obeyed are different properties and only the first one shows up in a normal eval. Telling the model to double-check its trace changed nothing — 12.9% evidence-following against 14.1% — but re-running the reasoning over the new evidence reached 86.6% for roughly double the tokens. Ordering carries much of the effect: putting the evidence before the trace in that separate call hands back 60% to 91% of the gain, so if you retrieve after you reason, answer in a fresh call and put the retrieved evidence last.

open the explainer → abstract PDF

72.4% vs 64.0%

terminal tasks passed when the critic probes a disposable copy of the finished state, against tree search whose critic only reads the transcript

Probe-and-Resume: When and How to Evaluate Agent Execution

Wrapping a terminal agent in tree search and scoring its half-finished trajectories is mostly wasted money: a quarter of the way through a run, a critic reading the transcript told winners from losers 49% of the time, so the whole search bought 0.6 points for 3.8 times the cost. What worked was moving the check to the one moment it is decidable — the agent says it is done — and handing the critic a throwaway fork of both that state and the starting state, with permission to run commands, delete files and install things while it tests each requirement in turn. Misjudged finished states fell from 36-39% to 22%, and the pass rate went from 63.4% to 72.4% over 89 tasks at $23 a sweep against $41 for the search. Rejection is a turn in the conversation rather than a verdict: of 136 failing first claims the critic caught 103 and the agent repaired 43, and of the 60 passing claims it wrongly rejected the agent recovered 57, so put the verifier at the done boundary with a disposable copy of the environment and let the agent carry on from where it stopped.

open the explainer → abstract PDF