It caught nearly every error.
It also binned a third of its right answers.

One executor, one revision, four occupants of the reviewer chair

The loop is the one most teams already run. The executor answers. A reviewer sees the problem and that answer — never the correct one — and either accepts it or rejects it with a critique. On a rejection the executor revises once, and the revision becomes the final answer. On an acceptance the original stands.

The executor solves 52 of the 100 problems unaided. Change only who sits in the reviewer chair and watch where the final number lands.

Only one of the three reviewers moves the number, and it is not the one that finds the most mistakes.

Catching errors and improving answers are different jobs

Split every rejection by what it was aimed at. Recall is the share of genuinely wrong answers the reviewer flagged. The false-rejection rate is the share of already-correct answers it flagged as well. A reviewer that rejects more of everything scores well on the first and badly on the second, and only the second costs you anything.

Self-review is the most sensitive detector in the study and the least useful one. It rejects 59 of 100 answers where the cross-family reviewer rejects 28 — 2.1 times as often — and it is picking up the executor's own uncertainty rather than the executor's own errors. The reviewer below the capability floor is the mirror image: two rejections, no false alarms, and no effect on anything.

Follow every rejection to the answer it produced

A rejection is not a correction. It is an instruction to try again, and the paper traces every one of them to the final answer that came out. Five outcomes: a genuine repair, a damaged answer, a wrong answer that stayed wrong in the same way, a correct answer that survived because the executor ignored the critique, and a miscorrection — a wrong answer replaced by a differently wrong one.

The repair rate is where the two reviewers actually differ: 43% of the cross-family reviewer's rejections produce a correct answer against 15% of self-review's, a gap that clears significance at p = 0.0074. Both reviewers are followed by the same executor doing the same revision, so this is not a difference in repair skill. It is a difference in what they chose to reject.

Self-review looks safe because the executor ignores it

Self-review damaged only 3 answers, which reads like a gentle reviewer. Follow the 18 correct answers it rejected instead. The executor left 15 of them alone and returned the identical correct answer. It complied on 3 — and all 3 became wrong. Its compliance rate on its own false alarms is 17%, and every single act of compliance was a loss.

So the safety is not a property of the reviewer. It is revision inertia, and it is not something you would design in. Drag the compliance rate and watch what a more obedient executor does to each pipeline.

At the observed 17% the self-review pipeline lands on its measured 58. At full compliance it lands at 43 — eleven points below running no reviewer at all. The cross-family pipeline barely moves across the whole range, because it only ever put one correct answer at risk. Same executor, same revision step: the difference is entirely in which answers got rejected.

Price a reviewer per correct answer, not per token illustrative

Reviewing is not free: the loop pays for the review and, on every rejection, for a second attempt. The cross-family reviewer is the most expensive per task and the cheapest per correct answer it adds. The reviewer below the capability floor is the pure-loss case — it doubles token spend and changes nothing at all.

What the paper actually reports