Swap the judge model on a checklist eval and the verdict barely moves. Swap the checklist and it moves almost all the way: with one judge held fixed, six hand-written checks passed 96–100% of known-good outputs while twelve auto-generated ones passed 2–8%. Then the part that should change how you run evals — asking a model to improve its own rubric made it worse in every arm the authors tried, because it deletes the checks that do the catching and adds ones that fail correct work. Under all-must-pass grading each check's exact share of a failure has a closed form, so you can see which is which in one pass over the answers.