A context window twice the size of its cache
Cloudflare served 155 responses carrying other customers' data because the cache behind one model held two thirds of the context window it advertised. Epoch scored frontier models at 65 per cent of its own staff rubric and calls that a failed task, and a poisoned LiteLLM release reached 49,000 downloads after tokens were read out of the project's own logs.
How we Organise
Sixty-five per cent of the rubric is a failed task
Epoch AI marked frontier models on the work its own staff does, using the rubric items a human employee is scored against. Claude Fable 5.1 and GPT-6 Astra led at 65 per cent. Epoch says every one of those outputs would have been rejected or revised had a person handed it in. One pilot experiment carries the sharper detail. GPT-6 Astra set the token budget too low for the models it was testing. 22 per cent of runs came back empty. It wrote those empty runs up as a finding about model capability. The setup was the fault.
We tell a client that somebody has to check what the machine produced. That somebody already had a job.
Reviewing machine output is work, and it is done by people who were doing something else.
Epoch’s own sentence is that these outputs still need a human to fix each mistake. The token-budget case says how good that human has to be. A reviewer checking format would have passed it. Epoch has also dropped one task from the evaluation. A model now does that one acceptably first time, and Epoch partly automates the step. That is what handing a task over looks like. It happens when the correcting stops.
How we Build
The cache behind the model held two thirds of the window it advertised
Cloudflare ran the GLM 5.3 model on Workers AI from 27 August to 9 September. The advertised context window was 1,048,576 tokens. The cache pool on its prefill GPUs had room for 681,280. Cloudflare knew about the gap. It expected the engine to reject anything that did not fit. SGLang did something else. It marked an over-length prompt as rejected, swapped in a one-token placeholder, and transferred that placeholder’s cache. The decode node then answered over pages the previous request had left behind. One hundred and fifty-five responses carried other customers’ data. Every one answered a prompt above 681,280 tokens.
We ask a team to write down what goes into an agent’s context. Also how old it may be, and what one finished job costs.
Each agent’s context is budgeted: which sources are admitted and in what precedence, how stale each may be, what survives compaction, and what one resolved task costs.
A team running that model could have filled in every field honestly. Its sources were listed. Their order was set. Somebody had judged how stale each could be. The cost of a resolved task was known. The budget is about the agent. What went stale was the platform’s cache. A vendor changelog confirmed this same position a month ago, so today is the second reading on it and the first from an incident. Cloudflare reports the gap itself, on one model, and says no other model on the platform was affected. Do you keep a context budget and treat the pool behind it as somebody else’s problem? Write to transform@dromologue.ai and tell us why.
How we Assure
The token was in a log before the release shipped
A malicious LiteLLM 1.107.0 reached PyPI at 11:39 UTC on 21 October. It took about 49,000 downloads before quarantine at 13:20. The maintainers say the release was triggered from one engineer’s GitHub account and approved from another. The credentials were stolen GitHub CLI OAuth tokens. Those came out of logs in an internal LiteLLM deployment that was reachable from the public internet. They also say Claude Code can put a user’s GitHub CLI OAuth token into a response body. Where bodies are logged, the token lands in the log.
Our position is that an agent carries a key of its own, narrow and short-lived, opening less at each step.
Every agent acts under its own scoped, short-lived credential, reaching no more than the person who asked, and authority narrows at each hop.
Nothing narrowed here. The credential was never the agent’s. A person’s long-lived token travelled through an agent’s reply into a log, and the log was the way in. A register of agent credentials could have been complete and current throughout. The reconstruction is the vendor’s and the maintainers’, published inside a day, and the route into the internal deployment is still described as suspected. The download figure comes from ClickPy rather than PyPI.