/p/2026-10-25 · explainer
Paper explainer · 2610.31118 · Wu, Yang, Du and colleagues

It built the change. It broke the page next door.

Give a frontend coding agent a real repository and one requested change, then grade it twice: once on whether the thing you asked for exists and works, and once on whether the rest of the site still does. Across 320 change tasks on eight frameworks and 8,737 runs from 13 models, 674 runs passed every check for the requested work — and 190 of them, 28.2%, failed a check on something else. 73 broke a page state the same run had already proved working. The unit of success is not the patch; it is the patch plus the site it lands in.

01 · Two gradebooks

pick a continuity check and see what each gradebook says

02 · The states you already proved

03 · The scoreboard

choose which column to rank the thirteen models by

04 · What actually moves it

step through the four levers the paper measured

weakest scaffold strongest scaffold

05 · Your own pipeline illustrative

Results

What the paper actually measured

What it does not show

In practice