Let the note-taker look things up

Three papers on artifacts an LLM system produces that nobody checks against the running system: the notes it keeps between sessions, the patch it says is finished, and the context it carries.

Today's pick

39% → 73%

database tasks the agent passed, memory written from the transcript alone versus memory checked against the live system

Grounding Agent Memory: Environment-Probing Curation for Enterprise Agents

Agent platforms now keep memory across sessions: when a task closes, a second model reads the transcript and writes down what it learned. Working from the transcript alone, that writer keeps whatever went wrong — it records the wrong average and its answer instead of the query that would have got the right one, promotes a single observation into a rule, and keeps a column name the schema dropped last month. This work gives that writer a read-only copy of the agent's tools and makes it check each note before committing: confirm the join holds on another slice, re-query when the world may have moved. Pass rate went from 39% to 73%, queries from 8.8 to 4.7 per question and cost from $3.38 to $1.68, with the agent, the retriever and the memory format unchanged.

open the explainer → abstract PDF

170 of 1,030

agent repair runs that compiled and passed their tests while leaving or adding a security hole

When Passing Tests Hides Vulnerabilities: An Empirical Study of Silent Failures in Agentic Systems

A repair agent is graded on whether the patched code parses and passes its tests, which is exactly the bar a security fix can clear while doing nothing. Across 1,030 runs from seven agent frameworks on two vulnerability datasets, 170 patches cleared it and were still unsafe: roughly half left the dangerous call in place under a cosmetic edit — a length check added, variables renamed, the shell execution untouched — and a third introduced a hole the original code did not have, ten of those pasting a signing key or token into the source as a literal. Every one would have scored as a successful repair on a standard benchmark, and the reviewer role in the multi-agent runs almost never caught them, because later turns grade progress against the plan rather than reopen the decision that made the patch unsafe. Gate the diff on a static analyser and a secret scan, not the test suite.

open the explainer → abstract PDF

half the KV cache

context cache at serving batch size, with 47% faster decoding and under 0.7 points of answer quality given up

FlexComp: One Model for Every Ratio in Context Compression

Soft context compression squeezes a retrieved document into a handful of memory tokens the model reads in place of the text, but the squeeze is fixed when the compressor is trained and then applied to everything, so a dense contract and a boilerplate FAQ get the same budget and every ratio you serve is another model to train. Draw the budget at random per example during fine-tuning and one compressor works at any ratio; a two-layer head then reads the document — not the question — and picks its budget in a single pass, cacheable per document, which at serving batch sizes halves the context cache and adds 47% to decoding throughput for under 0.7 points of answer quality. The finding worth testing on your own corpus is that more memory tokens are not reliably better: on about one document in eight the single-token budget beat the 34-token one, because the extra room preserved misleading detail.

open the explainer → abstract PDF