The exact number was the wrong one
How we organise
Token reduction is not cost reduction (PointFive)
PointFive ran 2,908 paid Claude Code sessions to test whether tools that cut tokens cut cost. Three compression tools went into the agent, with unmodified Claude Code as the baseline, across 103 tasks, seven repositories and three models. Every cost came off the provider’s own bill. The measurement plan was fixed before the first session ran. The tool that removed 1.3 per cent of tokens saved 2.9 per cent, and the interval does not separate that from no difference. The build that removed 38.4 per cent of the text cost 6.8 per cent more per completed task. A third-party tool cost 46.4 per cent more. The authors put the ceiling for current compression tools at about 5 per cent. They disclose that they built one of the tools tested.
Finance: the token is the most precise number in an AI programme, and precision is why it gets treated as the cost. It is not the cost. An agent that loses material goes and re-finds it. It pays for the extra turns, so the saving on the meter comes back further down the same invoice. A number an organisation can read to four decimal places will beat a number it has to argue about. We have reviewed cases built entirely on the first number.
Take one agent workload and put its token count beside its cost per completed task over the same period. If the two lines do not move together, the meter is not the cost.
How we build
State of Open Models: Summer 2026 Observations (Hugging Face)
Hugging Face set the top 25 open-model repositories by downloads accumulated in 2026 against the top 25 by likes. Exactly one repository appears on both lists. Counting downloads inside the window rather than over a lifetime sharpens the split rather than softening it. No model published in 2026 reaches the download list, and thirteen of the twenty-five date from 2022. all-MiniLM-L6-v2 was pulled 1.55 billion times in seven months against 5,156 likes. Across the whole hub, 1.5 per cent of repositories account for 99.2 per cent of downloads. The authors say the two numbers record different acts. A like says a release matters. A download says something is wired into a pipeline that runs on a schedule.
Architecture: the model an organisation depends on is rarely the model its coverage is about. Attention goes to whatever shipped last. Dependence accrues to small stable models over years, so a review that reads the first number selects for novelty and calls it a standard. The report names this as the most common mistake in coverage of the hub, including its own earlier work. An architecture decision taken on likes buys the release everyone discussed, while the pipeline still runs on something from 2022 that nobody has written down.
List the models your pipelines actually call this month, ordered by call volume. Compare that list with the models named in your last architecture review, and treat the gap as your real inventory.
How we assure
AGENTSEC02-BP01: Implement tool authorization (AWS Well-Architected)
AWS’s Agentic AI Lens carries a control on tool authorisation, and its anti-patterns are the useful half. It names reliance on the agent’s own judgement about whether a tool call is appropriate, with no independent check at the tool or API layer. Another is failure to carry user identity through tool calls, so every call runs with the agent’s permissions and downstream services cannot apply their own access controls. The third is skipping human review for mutating operations, because review adds latency. The desired outcome is that every tool call is authorised against policy before execution, with agent identity and user context carried through the chain. AWS puts the risk of not establishing this at high. The control opens by saying that an agent with unconstrained tool access has no meaningful privilege boundary.
Security: a valid token proves who is calling. It does not prove the call still serves the purpose the authority was granted for. That gap is where prompt injection does its work. The principal can be exactly who the directory says, the token can be valid and the permissions can be legitimate, while the purpose has been replaced. Authorising the tool call outside the reasoning loop is a refusal to let an agent decide the limits of its own authority. An organisation holding the identity layer and not the policy layer has bought the proxy rather than the control.
Take the agent with the widest tool access and ask where its tool calls are authorised. If the answer is inside the agent, you have logging rather than a boundary.