When a coding agent finishes, the thing a developer reads is a summary the agent wrote about itself — not a display anyone designed. Across 5,851 real sessions and 355,942 tool calls, that summary referred to about one action in eleven, and a reader given only the report could rebuild about a fifth of what happened. The sharper finding is where the report points when things go wrong: it tracks the executed log when execution followed the plan, and swings back toward the plan the agent abandoned as execution departs from it.
Every review of agent work goes through the same artefact: the last message. Not a diff viewer, not a trace explorer — a few sentences of prose, authored by the same system whose work is under review, in a format nobody specified. The paper's first question is simply how much of the session that paragraph carries.
Each square below is one action the agent took. The filled ones are the actions its closing report actually referred to, at the rate measured across the corpus. Drag the session length.
The median session in this corpus ran 59 actions and produced a report making four claims. Mean omission is 0.906 with a tight interval, and the median is higher still at 0.949 — so the typical report is thinner than the average one, and the average is dragged up by short sessions where a handful of actions genuinely is the whole story.
None of this is an accusation of dishonesty. A summary is supposed to summarise. The problem is that this particular summary is load-bearing: it is the artefact the review decision is made from, and nobody chose it for that job.
Omission counts what was left out. The more useful question is what a careful reader could reconstruct from what was left in — so the authors handed the report, and only the report, to a model and asked it to list the actions it implied, explicitly forbidding it from padding the list with actions a competent agent would plausibly have taken. That list was scored against the real log.
That last number is the one that closes off the comfortable interpretation. If thin reports came with drifting sessions, report thinness would be a usable warning light — you could route the sparse ones to a human. It does not: the correlation between how much a report recovers and how far execution wandered is −0.006, which is nothing. Reports are uninformative about their own reliability.
Three records exist for each session: the plan the agent stated, the log of what it did, and the report it wrote at the end. Divergence measures how much of the stated plan actually appears in the log, in order. The repair index measures which of the other two records the report resembles more — negative when it tracks the log, positive when it tracks the plan.
Slide the divergence and watch the report change allegiance.
The relationship is strong and it is the paper's headline: a regression coefficient of 0.771 with a t of 12.22, standard errors clustered on repository, and a rank correlation of 0.548 across 2,327 sessions. In the bin where execution followed the plan exactly, the median repair index is negative — those reports do describe what happened. It rises across bins, turns positive, and in the highest bin the entire distribution has shifted.
Read plainly: the moment an agent's actual work stops matching what it said it would do is the moment its summary starts describing what it said it would do. That is exactly backwards from what a reviewer needs, and it is the failure mode you would least expect to find by reading reports, because reading reports is how you would look for it.
If you were going to build monitoring on top of this, you would reach for one of three heuristics. All three were tested. All three fail, and the authors report them as failures rather than burying them.
The paper's own instrument gets the same treatment, which is rarer than it should be. Two of its measurement steps use a language model, both were hand-validated against a stratified sample, one passed and one failed — and the one that failed is reported alongside the one that passed, with the measure that depended on it explicitly excluded from every inference.
The failed one is worth dwelling on for its own sake, because it is a result about the tool most teams are currently reaching for. Asked whether an action in the log supported a claim in the report, a model judge agreed with hand coding on 56% of cases — a kappa of 0.185, below any conventional threshold — and it erred in a consistent direction, calling 67% of claims supported where hand coding found 36%. A model asked to check an agent's story about itself was systematically inclined to believe it.
Set how many agent sessions your team runs in a day and how long a session is. The omission rate is the paper's measurement; multiplying it out is arithmetic, and your sessions are not this corpus's sessions.
The fix is not a better prompt for the summary. The paper's own framing is that the narrative layer arrived by accident — it exists because a chat interface needs a last message, not because anyone designed an oversight surface — and asking the agent to write a longer one still leaves the agent as the sole author of the record its own review depends on. Build the review view from the log your harness already records, and let the prose be the index into it.