Hand a model a file and ask whether it has bugs, with no failing test to anchor the question, and it will find some. Across two models and 800 competitive-programming submissions, one review round was far more likely to break a program that already passed than to repair one that did not — 29.3% against 6.2% for the stronger model writing search-and-replace edits, and 42.4% against 4.1% for the weaker one. Iterate that blindly for a hundred rounds and it does not converge on correct code; it converges on a cycle, adding and removing the same change forever. Probing the activations found a single direction that governs how eager the model is to edit at all.
The setup is deliberately blind. A model is shown one C++ submission and asked to fix any bugs, with no failing test, no error message and no statement of what "fixed" would mean. Its edit is applied, the program is run against the problem's tests, and the result goes back in for another round — a hundred times.
That gives two numbers per configuration. The repair rate is the chance a failing program comes back passing. The damage rate is the chance a passing program comes back failing. Pick a model and an edit format, and switch the sampling temperature.
Every configuration in the paper sits the same way round: the damage rate is several times the repair rate. On the weaker model writing structured edits it is ten times. There is no setting here where a blind review round is a good bet.
Those two rates are transition probabilities, so the loop has an arithmetic destination. Whatever share of your files starts out correct, each round moves some correct files to broken at the damage rate and some broken files to correct at the repair rate, and the share settles where the two flows balance — repair rate divided by the sum of both. Nothing about running more rounds changes where that is.
Start from a clean repository or a half-broken one, drag the round count, and watch it forget where it started.
The destination is somewhere between one program in forty and one in five, depending on the configuration — and it is the same destination from either starting point. A loop with no stopping condition is not a repair process. It is a mixer.
Following the individual runs to their ends, the paper finds two kinds of resting state. Some runs reach a fixed point, where the model finally stops changing anything. Many reach a cycle instead — a pseudo-bug-fixing loop where the same edit is applied and reverted forever, sometimes bouncing between a passing and a failing version of the same file.
Which of those you get depends on the edit format, and this is the one operational choice in the paper that clearly matters. Search-and-replace blocks — one small targeted edit per turn, the format almost every coding agent uses — produce more cycles, longer cycles, and longer runs before anything settles than rewriting the whole file.
A small diff looks safer and behaves worse. The whole-file rewrite makes the model restate the program, which at least forces it to reproduce the parts that already worked; a search-and-replace block lets it nudge one line without ever looking at the consequences, which is precisely the move that can be undone next round and made again the round after.
If the model keeps volunteering fixes to correct code, something inside it is representing that code as buggy. The paper goes looking: it takes the activations at the last token before the model answers, averages them separately over programs the model called buggy and programs it called clean, and subtracts. That difference is a single direction in activation space, read off at layer 22 of the network.
A probe trained on those activations separates the two classes almost perfectly from the middle layers on, so the representation is there and linear. Then they add multiples of the direction back into the running model and watch the editing behaviour move.
Turning the direction down stops the pseudo-fixing entirely and preserves every correct submission — and repairs nothing. That is the honest summary of blind repair: the eagerness that damages working code is the same eagerness that occasionally fixes broken code, and there is no setting of it that only does the second thing. What separates the two cases is information the model does not have, and that you do: which files are actually failing.
Point the paper's rates at a review bot. Set how many files a week it looks at and how many of them are actually broken, then gate it: only let it edit a file that has a failing test. The gate does not make the model better — it removes the population the damage rate applies to.
Ungated, the arithmetic is brutal in a way that is easy to miss in review: the damage lands on files nobody was worried about, so it arrives as a regression with no ticket attached, while the repairs land where you were already looking.