The person watching said yes
How we Organise
What each part of a pipeline actually earns, priced separately
Lin and eight colleagues published this on 28 August. Papers on text-to-SQL usually report one accuracy figure for a whole pipeline, which tells a reader that the pipeline works and nothing about which part of it did the work. They built 17 configurations across the five modules that recur in these pipelines, held the implementation constant, and measured each module’s marginal accuracy and its cost on four backbones of differing capability and reasoning style.
Execution-feedback refinement was the only module whose benefit held on every backbone at consistently low cost. Every other module helped only under conditions the backbone set, so a module that earns its cost on one backbone earns nothing on another. Token accounting separated two demands that one accuracy figure hides. Input volume tracked the pipeline, and output volume tracked the backbone. Stacking modules raised accuracy on most backbones, though how the gains combined varied with capability. Their result about where to put a fixed budget is the one that matters. A fixed budget usually earns more accuracy from an elaborate pipeline on a mid-tier backbone than from a lean pipeline on a frontier backbone. Their tiered guideline held on five more backbones, so nobody had to measure each module again.
We tell a client to work out what one successful outcome costs it, computed from its own costs and its own test results rather than assembled by hand. This supports that, and it shows the shape of the error the alternative makes. A team reading an aggregate accuracy figure off a public leaderboard cannot see that most of the modules it pays for earn nothing on the backbone it happens to run, and it will spend the next increase on a better backbone when the same budget earns more inside the pipeline.
One task family carries the whole result, chosen because a query either executes correctly or does not, and four backbones is a thin population to generalise a tiered guideline from. The costs are token costs. The engineering time to build the more elaborate pipeline sits outside the comparison, and that is plausibly where the mid-tier saving goes.
How we Build
Inside a Claude Code skill, the prose and the code move together
Hereiz and four colleagues published this on 28 August, from 1,926 repositories hosting Claude Code plugin marketplaces. That covered 8,351 plugins and 77,773 commits across 2,018 marketplaces. Plugin-touching commits grew 8.8 times over the six months after the October 2025 launch. Plugins aimed at software engineering were 61.3 per cent of the total. Development is feature-driven at more than twice the rate of conventional open-source work, 39.6 per cent of commits against 17.2, and Claude is a co-author on 34.9 per cent of them.
Their finding about maintenance is the one that matters. Most component types evolve on their own.
Inside skills directories they do not. Instruction files written in prose and the scripts that implement them co-evolve above chance, and of those co-changes, 78 per cent are functionally coupled. They call this a class of maintenance dependency that traditional software engineering does not show.
A team following us enters every skill and every agent it builds in a catalogue. Each entry carries an owner, a version and a status. It declares what that skill depends on, and whether the dependency has promised notice before a breaking change. Every dependency in that catalogue runs between two catalogued things.
The coupling measured here runs inside one of them, between a skill’s prose and the code that prose describes, and that is where 78 per cent of the functionally coupled changes were. So a team can hold a complete catalogue and declare every dependency the catalogue is able to express. It has still declared nothing about the joint most likely to come apart. We may be wrong about how much that matters. This is one ecosystem, ten months old, and a marketplace plugin is not a production skill inside a firm. Co-evolving above chance says two files changed in the same commit more often than chance predicts, which is weaker than saying one broke the other, and nobody reports a failure caused by an uncoupled change. A young ecosystem finding its conventions would look much the same.
If it stands, a catalogue entry has to declare the coupling within a skill as well as the ones leaving it, and say which components inside it move together. Until then we stop telling a client that a complete catalogue with its dependencies declared is evidence its agent estate is maintainable.
How we Assure
People who could refuse approved almost all of it
Ting Yan published this on 27 August. 113 participants with no professional software background each supervised an agent through an 18-action simulated day, seven of whose actions reached beyond what had been asked for. They did it under one of three regimes: approving each action as it came, letting a model review each action, or writing their own plain-language rules of allow, ask or never against four categories of consequence.
Writing the rules in advance was the worst of the three. It blocked 20.1 percentage points less overreach than approving action by action, and 14.5 points less than the model reviewing each action. It did cut runtime prompts from 18.0 to 10.9, but total intervention time was not reliably lower once the rule-writing was counted. Participants chose ask for 114 of their 140 rules. That handed most of the overreach back to the moment it happened. Then the number that carries the paper: of the 148 overreaching actions that ran under the rules, 133 followed a human approval, and only 15 ran automatically under an allow rule.
Our position is that every agent and every automated decision has a named person who can stop it, and that stopping it has been exercised. Both halves hold throughout this study. Each participant was that named person, each could refuse, and refusals happened. The exposure the position exists to bound stayed open anyway, because a person shown one action at a time approves most of what they are shown, including the actions nobody asked for. Yan names the gap as one between preference and commitment. Choosing ask again and again preserves the choice and stops any standing rule from settling anything.
We may be wrong to read this as a limit on human oversight. The participants were not technical and the day was simulated. There was no cost to approving wrongly, and no prior relationship with the agent to draw on. Seven overreaching actions in eighteen is a denser stream than a real day, so fatigue cannot be separated out, and a reasonable person could call this an interface problem that a better prompt would fix.
If it holds, exercising the stop has to be measured against actions the person did not request, rather than counted as an event that occurred. We stop telling a client that a named person with a stop button, and a record showing the button has been pressed, is evidence its agents are contained.