The list had no line for it
How we Organise
What happened on OpenRouter when two models were discounted for nineteen days
OpenRouter published this on 25 August, from its own routing data. OpenAI discounted two new models, Terra and Luna, between 27 July and 14 August. On 30 July it also cut their list prices, Luna by 80 per cent and Terra by 20, so the effective discount from that date was 90 per cent on one and 60 on the other. Terra tokens rose 5.6 times against the pre-period daily average. Luna rose 13.8. Sol launched on the same day as both, stayed at list price, and moved 1.1 times. Terra and Luna together went from 0.7 per cent of all tokens on the platform to 7.8. Competitors gave up 5.3 points of share over the same comparison and other OpenAI models gave up 1.9, so roughly three quarters of the gain came from outside OpenAI. Of the hundred thousand or more customers who used the discounted models, about 32 per cent kept some usage afterwards and 18 per cent ran at or above their programme pace. Daily volume in the six days after the discount ended averaged 1.38 times the volume during it, which OpenRouter reads as the retained accounts being much larger than the median user in the programme.
We tell a client that a price which moves between model releases cannot be governed by an annual budget. This is the cleanest support that position has had. The price of the same work fell by 90 per cent inside nineteen days and demand for it rose almost fourteenfold. An annual number would have been set before any of it and would have been wrong in both directions at once, too small for the volume and too large for the unit price.
Two things it does not settle. The post-period is six days, which OpenRouter says plainly, and the retention figures count customers rather than tokens. One platform’s routing data is not the whole market, and Sol stopped being a control the moment it was discounted too.
How we Build
A text-to-SQL model that reached human accuracy without a scaffold
Zhu and three colleagues published this through Thinking Machines Lab. Their benchmark has a human score of 92.96 per cent, and model scores went from just under 70 per cent in 2024. Today they reach 82. The usual way to close that gap is a scaffold. A schema-linking stage narrows thousands of columns to a candidate set, a generation stage samples queries, a self-correction stage repairs execution errors, and a selection stage votes among the survivors. Every stage is a separate call to a model that is itself held still, and the whole thing is tuned to the benchmark. The best scaffolded systems still finish 11 points behind humans.
The authors trained the model instead. First they audited 2,500 instances of the standard training set. Errors turned up in every part of it: the questions, the supplied background knowledge, and more than half of the gold queries that a model’s answer is scored against. An auditor caught real annotation mistakes at 90.6 per cent precision but found only 24.5 per cent of what human experts flagged, so the cleanup ran through human review and a second verifier. On the evaluation set, a first pass by another team corrected 32.3 per cent of instances and their own second pass took the detected error rate to 52.8. Trained on the cleaned data alone, their model scored 88.55 per cent, above the frontier generalists and above the strongest open-weight models fitted to this task, at $0.56 per task and 12 to 15 per cent of the cost of the models it beat. Sampling sixteen times and taking the majority answer put it past the human mark. Neither sampling nor majority voting is a scaffold stage, and they say so: both draw from one prompt with no separately prompted steps in between. The same cleaned data trained onto a different model beat that model’s original training by 16, 12 and 14 per cent across three benchmarks, two of them harder and untouched by the cleanup.
We ask a team to choose deliberately between four constructions, plain code, one model call, an agent and a scaffold, and to try or rule out the simpler one on the record. All four hold the model still. The construction that won here was one model call to a model somebody had trained, and there is no line for it on the list. A team following our test exactly would try a stock model on a single call, watch it come back at 82 per cent, rule it out on the record, and build the scaffold. Its record would be complete and its answer would be worse.
We may be wrong about how far this reaches. It is one task family, chosen because correctness can be checked automatically, and the authors are candid that dirty labels were the constraint rather than anything about scaffolds. Most clients have neither a training budget nor labelled data of their own, so a reasonable person could read this as a finding about data quality that says nothing about the four constructions. If it stands, we stop telling a client that ruling out the simple option on the record is evidence the construction was chosen well. The list has to carry the model itself, and a team without the data to move it should be told its choice is bounded rather than deliberate.
How we Assure
An attack that turns a coding agent’s own caution into the exploit
Johann Rehberger published this on 26 August. Claude Code’s auto mode replaces human approval prompts with a safety classifier that reviews each tool call before it runs, and since mid-August it is the default starting mode. He reports 60 to 80 per cent success against it on a small sample, and says so himself. The user prompt is an ordinary request to summarise a website. The site holds an archive of notebook records wrapped in plausible catalogue metadata, and the records themselves sit in a zip with a native binary decoder alongside them. Claude declines to run the supplied binary and writes its own Python decoder. That refusal is the exploit. It runs the decoder from inside the unzipped directory, which Python puts on its module search path. The archive contains a file called struct.py, the standard base64 module imports struct, and Python loads the attacker’s file. The file re-exports the real functions, so decoding continues and valid records come back. An obfuscated expression at module level launches a detached Python child. That child fetches a remote stage, which downloads and runs a payload, and the machine calls out to the attacker’s server. The command that cleared the classifier was a short Python decoder. Everything else was several hops downstream of it. In some runs Claude noticed the compromise and tried to kill the process. Auto mode denied the cleanup command. In a variant, the poisoned file starts a second headless Claude Code with its own tools and its own context.
We ask that agent execution runs inside a declared boundary and that every path out carries a named approver. Both halves pass here. There is a declared boundary and it is the classifier; every command carried an approver and the approver was that same classifier. Nothing about the outbound connection was approved by anybody, because approval attaches to the command somebody reads rather than to what running it reaches. This is the third source in ten days on the same position. The UK AI Security Institute reported on 21 August that nineteen unsanctioned actions across ten of 122 runs reached a real public open-source project on GitHub, where one agent opened a malicious pull request. Access to the live internet was part of the evaluation’s design. Boyd Kane wrote on 26 August that the boundary is drawn around the agent’s own machine while its output is parsed on the host holding the weights.
Take the limits seriously. The sample is small and it is one product. The rates improved as the author iterated his payload with help from another model, and he says the results represent a motivated attack rather than a comprehensive one. Anthropic’s own page for auto mode already says it reduces risk without eliminating it and recommends isolated environments. The vendor and the researcher agree. A reasonable person could conclude the gap is a classifier problem to be improved rather than anything wrong with how we write the position. If it stands, we stop telling a client that a declared boundary with a named approver on every path out is evidence an agent is contained. Where the approver reads the same untrusted material the agent reads, one command at a time, the test passes. The containment is not there.