Eighty-three real security-audit skills, collected from skill marketplaces and public repositories, run over 40 real audit tasks holding 120 known bugs. One configuration opened a skill in all 40 runs and found 22.8% more of the bugs, worth 43.2% more in bounty value, on 5.2% fewer tokens. Three configurations opened one in none of their 40. Put a weaker model behind the same three harnesses and the effect turns negative everywhere, from −5.9% to −9.4%, with the spread between harnesses staying inside 11%. The backend model decides whether a skill is worth anything. The harness barely registers.
A skill is a folder with an instruction file at the top and whatever it needs beside it — reference material, report templates, scripts the agent can run. The agent is supposed to notice when the folder is relevant and pull it in. That is the entire mechanism, and it is worth seeing how modest the artefacts are before asking what they do.
The median skill here is 1,281 tokens across 120 lines and three files, comfortably under the 500-line guidance everyone repeats. Step through what those lines hold.
Across the corpus 2,482 vulnerability items map to 37 categories, with a median skill covering nine of them — reentrancy and arithmetic precision are in almost half the files, and the protocol-specific risks trail off into a long tail. The coverage is broad and uneven, which matters less than the next question.
Every configuration below had the same 83 skills sitting in the same place, and the same 40 audits to do. The only thing that varied was the agent and the model behind it. Step through them.
Three configurations never read a single skill in 40 audits each. They still ran the audits, still produced reports, and still moved a couple of points on the score — which is exactly the trap. A measured difference between a with-skills arm and a without-skills arm is not evidence the skill did anything if nothing in the trace shows it being loaded.
The one configuration that triggered reliably pulled in 27 distinct skills over 119 reads, chaining a different one into each stage. The next best used two skills over 31 reads. Everything below that is noise.
Forty audits, 120 annotated bugs, and the score is how many of them come back. Flip the skills on.
Bugs found is the friendly metric. Bounty value is the one that pays, because the annotated bugs carry their real award labels and finding three cheap ones is not finding one expensive one. The configuration that triggered reliably went from $108,241 to $154,983 of the available pool and got more precise at the same time. The one that triggered in a third of its audits nearly doubled its award while its precision slipped. The two that barely triggered moved backwards on money.
The clean experiment is holding the model fixed and changing everything around it. Three harnesses, one weaker backend behind all of them, the same skills, the same tasks. If the harness were doing the work, the three would separate.
They do not separate. On the shared weaker backend all three land within eleven percent of each other and all three go down when skills are available — the model spends context on instructions it cannot act on. Whatever you were hoping the harness would do for your skills, it does not do it.
The trigger rates and the uplift below start from the paper’s measurements; the size of your library and the rate at which your agent actually opens things are yours, so the totals are labelled illustrative. The point of the arithmetic is that trigger rate multiplies everything else.
The cost column is the part people assume and should not. Skills are not free context: on the configuration that triggered reliably they ran cheaper and faster than no skills at all, because the agent stopped flailing. On two others tokens went up by roughly twelve and forty percent. Same corpus, opposite bills.