The record looked right
How we organise
Can Agents Use a Computer Yet? We’ve Got the Data (Fabrizio Serafini, Seema Amble and Eric Zhou, a16z)
a16z spoke to teams running agents that drive a desktop in production. It published what they found on 10 August. A year ago the best model scored 42 per cent on the standard desktop benchmark. The best today scores 85 per cent, against roughly 72 per cent for human testers. One data platform runs 15 to 20 million portal interactions a month. Agents repair the hand-coded scrapers that break whenever a retailer changes its pages, and the platform halved the engineering team that used to do those repairs. A systems integrator runs 27 workflows handling between 1,500 and 2,100 IT tickets a day. The authors are exact about where this stops working. An agent that reads net 60 as net 30 writes a record that looks perfectly plausible. It passes every check a person would run by eye, and surfaces when the invoice goes out wrong.
Structure: the work that can move to agents is the work that produces evidence somebody can check. That is a different question from whether the model can do the job. Both failures a16z names are failures of checking. The first has no cross-check, so a wrong record reads exactly like a right one. The second has no signal while the work runs. The truth arrives two days later, as a phone call to somebody’s desk. We sort candidate work by where its check comes from, before looking at any model. Where a person reads every output, no work has moved: reading it costs what doing it cost.
Take the work your organisation most wants to hand to agents this year. Write down how anyone would know that a single run of it went wrong. If the answer is that somebody notices later, you have a checking problem, and no model release will fix it.
How we build
Introducing Muse Glimmer: An Open Agentic Model That Runs on Your Device (Meta Superintelligence Labs)
Meta released Muse Glimmer on 10 August, a 30-billion-parameter agentic model. The weights are published under the Apache 2.0 licence. At full precision the model would need more than 55 GB of memory. Meta compresses the weights to roughly 4-bit precision, which brings it under 20 GB. That leaves room for its working memory and its image encoder inside a 24 or 32 GB envelope, so it runs on one consumer GPU. The compression introduces minimal to no degradation on agentic work, according to Meta. Muse Glimmer was trained by distilling a much larger teacher. It is trained to diagnose a failed tool call and retry rather than stop.
Architecture: where an agent runs is a choice again. For two years it has not been one. An organisation that wanted a capable agent bought network access to somebody else’s data centre, and that single decision set its cost, its latency, its data path and its regulatory position together. A 30-billion-parameter model on a laptop is not a frontier model and will not do frontier work. It is enough for the standing, repetitive work that runs all day. The volume sits in that work rather than in the hard cases. Our first question now is which agents need the frontier and which need only to be near the data. A buyer who has never asked cannot answer the first question a regulator asks, which is where the data went.
List the agents your organisation runs continuously rather than occasionally. For each one, work out what it costs a month and what leaves the building when it runs. The candidates for local work are the ones where both answers are uncomfortable.
How we assure
Learning more about Claude’s mathematical capabilities (Anthropic)
An unreleased research version of Claude raised a longstanding lower bound on the zeros of the Riemann zeta function. It moved from 41.6 per cent to 67.2 per cent, and Anthropic published the account on 10 August. The result took two sessions and 31 million output tokens. The first 650 ideas the model tried all failed. What Anthropic describes after that is mostly checking. The model coordinated about 60 subagents, of which 13 did nothing but check the arguments the others produced. It ran thousands of numerical checks against known zeros. It downloaded 54 papers to establish that nobody had reached the result already, and subagents re-proved the finding from scratch. The model then recommended that a human number theorist look at the work. Two mathematicians read the paper, two outside experts examined it, and a machine-checkable proof produced by the model passes the standard tool.
Risk: the result counted because of the checking attached to it. That checking is the part an organisation can copy. Nothing here rests on trusting the model. Every claim is settled by something outside the thing that produced it: other agents hunting counterexamples, a literature search, a human expert, and a proof a machine can check without reading it. That is the shape of an agent deployment that can be relied on. It cost roughly a fifth of the agents in this account. Checking goes in our budget as a line of its own, not as a read-through at the end. An organisation that budgets for the work and not for the checking has bought output it cannot use.
Take one agent already running in your organisation and count what checks its output. If the answer is a person reading a summary at the end, you have the one arrangement this account never relied on.