The unit price is not the bill
Unit prices falling while bills rise, a scheduler that doubles throughput without touching the model, and the best model on a benchmark behaving worst on it.
How we organise
Price per token falls. Tokens per task rise faster. An agentic workflow that plans, retries and reads its own transcripts consumes an order of magnitude more than the completion the pricing page assumed.
We tell a client that spend rising while unit prices fall is not a procurement problem. Changing model will not fix it. The lever is context: how much the system carries, how often it carries it, and who decided that.
Why compute might get ten times more expensive
Lab revenue is targeting tenfold annual growth while compute grows about threefold. Margins, the inference share, or the price of compute has to close the gap. The author calls the piece a two-hour experiment.
Our reading is to take the structure and leave the headline number where he put it. Every multi-year AI case assumes the price of compute falls. Nobody writes down what happens if it inverts.
How we build
The binding constraint is cache thrashing, not capability. Schedule for the access pattern rather than for chat and throughput goes from 390 to 803 tokens a second.
Our position is that a speedup which widens with the cluster is not a benchmark artefact. It is a different cost curve. Anyone modelling agent economics on today’s tokens per second is modelling a number a scheduler can double.
Base input tokens on a default agent turn fell 65 per cent. The release notes refuse to call that free money. Reward intervals span zero for every model, and one came out more expensive.
We read this as the difference between “no worse” and “not measurably different”, which almost every vendor collapses into one claim. The tokens go. Whether the quality went with them sits below the resolution of the instrument.
Self-improving memory for agents
Reported gains of 25 per cent on answer correctness for repeat tasks. Vendor numbers.
We read the direction rather than the magnitude. Memory is justified here as a cost lever, because a system that remembers does not pay to rediscover.
The answer to the harness question
A harness is two things ageing in opposite directions. The how rots as models improve. The what, meaning intent and constraints and taste, appreciates.
We hold that no lab can post-train your context into their model. It has to arrive from outside every single time, and that gives a test for any line in an instruction file. If a competent model would get it right once it knew the missing fact, the line is durable. If it specifies steps and approvals, its shelf life is measured in model releases.
How we assure
The best model on the benchmark was also the worst behaved on it. It fabricated supplier quotes, claimed a shipment had arrived with the wrong items, proposed price cartels in all six arena runs and broke eleven of the truces it agreed. The authors call this anecdotal evidence of misalignment rather than measurement.
We judge the objective and the environment rather than the model. This is a benchmark that scores profit, run on a model told to maximise it, and collusion, fabricated evidence and a broken truce are all locally profitable. None of it would surface in an evaluation of the model on its own, because the misbehaviour is a property of the objective and the environment, which is to say of the harness.
Securing agents at the client endpoint
Fifty-two rules across eleven behaviour categories. They hook into agent harnesses and export telemetry an auditor could read.
We ask where a firm’s agent controls sit: in the model, in a policy, or at the endpoint where the credentials are. Only the last is in the same layer as the failure. This one arrived from a competitor rather than from the lab whose agents caused the incident.