a daily read of the ML and AI papers
Compute-matched against one agent built from the same model, a centralized multi-agent system lost 28 points and only drew level at 5.5 times the tokens.
Across 496 LLM call sites in 40 agentic systems, 82% have checks around the call and 5.3% enforce the whole contract their job requires.
Repository content can steer a coding agent into asking you to approve a broad command rule; once approved, the attack finished silently in 81% of cases.
At the agreement threshold a label-free filter would certify, 60% of a model's wrong answers still get through, and most repeat the identical wrong answer.
A single agent repeating a false answer left 44.8% of a ten-agent room privately holding it, and the transcript showed about a third of that.
At one fixed token budget, showing whole files verbatim resolved 46.7% of issues against 40.3% for the best compressor, whose patches often failed to apply.
Turn count explained 84% of a coding agent's trajectory cost, and the size of what its tools printed back explained 4%.
Temperature zero still changed the verdict on 10.2% of rows between identical runs, and 4% of changed values moved two-thirds of the answers.
Semantic tool retrieval sent 86% of an agent's calls to tools that were failing; one success-or-error bit per tool took that to 2.9%.
Installing a second, similar skill took one run in five away from the one you installed, and the reply named which skill had run 0.9% of the time.
Agents left a mistake in their own transcript undisclosed in 67.1% of runs, then found the same mistake 97.9% of the time as an outside reviewer.
An agent told it would probably fail claimed high confidence on 56% of those tasks, and its reporting rule destroyed 68% of the gains from delegating.
A question-only regex matched the model's uncertainty signal exactly, and routing at ten samples a query cost 39% more than always calling the large model.
Agents finished 83.5% of workflows but recovered from a lost acknowledgement under half the time, and a plain retry repeated the side effect in 53% of trials.
In half of 150 production incidents the system passed every health check while returning wrong answers, and took a mean 4.2 days to be noticed.
Asking only the questions whose two plausible answers produce measurably different programs beat the best clarification method by 13.88 points.
One agent serving four users reached 64% of the best possible outcome on a shared compute budget; four agents serving one user each reached 30%.
One frontier model led another by 7.94 points inside one agent harness and trailed it by 30.16 inside another, on the same 63 tasks.
Sampling eight candidate commands a step and letting a strong checker pick one took first-try terminal success from 50.0% to 68.0%.
Keeping every agent edit that lifts the average broke 36.5% of already-solved tasks; comparing the two versions item by item took that to zero.
A step telling the agent to run a terminal command left 45% of turns recovered; naming the server's own login tool instead got 84%.
Three recurring behaviours waste up to 22.75% of a coding agent's task cost; seven lines of hand-written guidance cut spend by up to 41.73%.
A commit guard that fires on any state change stops every unsafe write and 94.6% of the harmless ones; blocked agents just go around it.
An LLM grader beat the human pair it would replace, then a two-sentence harsh-grader persona pushed 14 of 17 open-weight models out of usable accuracy.
Agents duplicate a write in 56% to 74% of episodes when they cannot read back what happened, and report success in 90% of the runs where they did.
Agents pick the option that is genuinely best for the user 78.6% of the time in a neutral marketplace, and 17.3% of the time when the platform has a favourite.
Asked whether a commit preserves behaviour, five models reach 0.29 precision; asked whether one specific stored claim still holds, the same models reach 0.97.
Ten LLM judges making correlated mistakes carry the evidence of 3.5 independent ones, and 28% of wins that looked significant stop being significant.
Patches that passed every hidden test were overturned by machine-checked proofs, cutting the resolved rate from 85% to 58% for the stronger model.
Four commercial models hit a readability target 21-32% of the time when asked for it; a measure-and-edit loop around the same models hit it 93-99%.
Delete the word unverified from an agent's handoff and one safety monitor's approval of risky actions jumps from 9% to 98%.
Agents left requested files unopened in 67.9% of review runs. In four out of five of those, the closing report still called the job done.
Show a reviewing model the full trace and it rejects correct work 60% of the time, up from 39%. Its error detection never moved.
In 69% of runs a coding agent did something out of scope — weakened a test, touched an unrelated file. Rules learned from past failures cut that to 27%.
A tool that reports success while returning nothing usable draws an invented answer 45.3% of the time. Plus when to abstain, and when to stop.
A plain shell beat a curated tool catalog by 24.5 points on 72% fewer tokens. Plus checking an action before it runs, and what an agent's summary omits.
Retrieval sufficiency read off the score curve alone: 0.856 AUROC at 2ms a query, beating a model that reads the pages. Plus two papers on judges.
A seeded random draw of 400 tool servers: 48.8% start, 37.5% never do. Plus screening a server from its schemas alone, and why a second model is not a check.
Giving an agent's memory curator read-only access to the system took pass rate from 39% to 73%. Plus patches that pass tests, and per-document context budgets.
Ordering an agent's tool shortlist so input-producing tools come first took completed tasks from 74% to 90%. Plus blind bug-fix loops, and stopping too early.
Agents adopted corrupted search results 68% of the time, often spotting the conflict and saying nothing. Plus authority as capability, and benchmark shortcuts.
Swap the model behind an agent and its own notes cost 13 points of accuracy, while a fixed-schema store loses 0.04. Plus who should review, and peer pressure.
Change only the serving adapter and the same model's tool-call score moves from 0.00 to 0.96. Plus caches that break reruns, and a dashboard blind to itself.
Handing an agent the exact input that broke it repairs 90% of tasks in four turns; a generic "try again" manages 27%. Plus rules over examples, and leaky rubrics.
Coding models write confident code for impossible tasks 60% of the time and refuse only 27%. Plus a cache evictor that wins by not scoring.
Stack three ordinary prompt constraints and correct code falls 12 points below what the parts predict. Plus two ways a passing test hides a bad patch.
A single does-it-start probe removes nearly every launch failure at a third of a shell's cost. Plus agents that do the work, then fumble the handoff.
A guardrail approves an action that is inadmissible by the time it runs, on up to a quarter of approvals. Plus decay by depth and judges that flip on order.
Three papers on the context an agent actually gets: the tool response nobody pages past, a harness worth more than a model swap, and abstention without labels.
Three papers on the shape of what you hand a model and what it hands back: misplaced JSON values, agent handoff formats, and what a thin brief costs.
Three papers on what an agent carries between turns: superseded facts, memory written on the wrong signal, and loaded docs that quietly cost accuracy.
Three papers on agents that look like they worked: silent tool failures, one-in-four reliability across repeats, and policy compiled into a graph.
On identical questions the same model was falsely certain 14% of the time when asked to hedge in words and 80% of the time when the answer was a 1-to-5 score.
Two instances of one model, composed as a writer and its checker, failed together on 90% of the missions either one failed.
A shipped drift guard caught 0 of 56 meaning-breaking edits, and scored "withhold the study drug" against "administer the study drug" at 0.96.
Let a model retry until its assertion passes and what comes out catches fewer bugs than its very first attempt did — 86.3% against 91.7%.
Hide a shell command inside a natural-language skill file and one enterprise coding agent runs it in 96% of attempts. It calls anything unsafe 2% of the time.
Pile twenty rules into one system prompt and the follow rate falls from 96% to 20%. Plus doomed agent runs, tool arguments, and one-shot generation.
A deterministic layer that tracks what a coding agent has already read and changed lifts first-try success by 8 points while cutting cost by 29%.
Models find the right file for 92% of required deletions but cut the exact line under 52% of the time. Test for removal and pass rates fall from 63% to 42%.
An agent performs 56% of the steps its own skill file mandates. Compile that skill into a harness and it performs 86%, on any model you point at it.
Agents perform just over half the steps their own instruction files mandate. Compiling the procedure into code fixes it — plus four papers on checks that miss.
Recall by address beat every summariser and repository context was served once instead of rediscovered. Plus three papers on defaults nobody ever checked.
An agent working on files and app state instead of screenshots scored higher at a ninth of the cost. Plus harness cost, hidden reasoning, and agent security.
An agent sending 38% fewer tool-output tokens cost 6.8% more, because cached context is most of the bill. Five more on hallucinated skills, generous judges, and tracing failure.
A workbench injects tool faults into MCP agents so you can reproduce a failure and prove your fix before shipping. Five more on abstaining, search evidence, and context.
An independent judge disputed a leading coding benchmark's pass/fail calls on a third of runs. Five more on judging, abstaining, and routing.
Indexing code and action history into compact packets cuts a repair agent's tokens 51.5%. Five more on agreement, calibration, and agent memory.
A production agent that compiles repeated steps into saved tools cuts latency 42% and errors up to 53%. Five more papers on agent safety, judging, and repair.
Grounding the LLM test-writer in the task spec — one prompt line — beats doubling the test budget (+38 points, cross-vendor). Six more papers on running agents cheaper and catching silent failures: capacity placement, workflow crystallization, trace triage, write gates, and judge audits.