trivago wants 600 people to work like 6,000
trivago's chief executive set out how the firm governs its agents, Thoughtworks asked 26 developers whether they trust the tests an agent writes, and the NCSC began assuring the people who attack your systems for you.
How we Organise
trivago publishes its 500 agents and skills in one marketplace so nobody builds a second copy
Johannes Thomas is trivago’s chief executive. He says the firm wants its 600 core staff to work with the impact of 6,000. Autonomous agents now ship complete code changes from start to finish on trivago’s own agent platform, and the human work has moved to directing those agents and judging what comes back. More than 500 agents, skills and components are published in an internal marketplace, each one qualified, so an employee re-uses it instead of writing another. In the rebuilt mobile app an agent wrote almost all of the new code. That took four months, against the twelve to eighteen trivago estimates it would have taken by hand.
We tell a client that an agent either belongs to a team that answers for it or runs without anybody watching. The catalogue is where the difference shows.
An agent is a member of the team, or it is running unsupervised.
trivago built the catalogue before it built the agents. That is the order this position asks for. What the account does not give is the supervision itself. Nobody is named against any of the 500. Qualified is trivago’s word rather than a stated test. September’s evidence against the position also still stands, because the Cloud Security Alliance measured a third state it does not name, where an agent runs under a shared or a human identity. The figures here are the firm’s own, taken from its coding tools, and a chief executive presenting to investors is not a neutral reader of them.
How we Build
Matteo Vaccari interviewed 26 developers at a Thoughtworks organisation working heavily with coding agents. He asked each of them whether they trusted the tests. Every one of them said not much. Their reason was not that the generated tests were bad when written. The agent refactors the tests along with the code as features arrive. A test reviewed in January is not the test running in March, and nobody rereads machine-written test code at every step. When an agent writes the code the developer is its customer. So the contract belongs in customer tests, and those tests move into data rather than code. There an agent editing an implementation cannot quietly edit them too.
We judge that the work gets stuck at checking rather than at producing. Cheaper production moves the queue rather than shortening it.
Verification, not generation, is the constraint.
This is not the first source to bear on that claim. A Microsoft study in August measured a sustained 24 per cent lift in merged changes per engineer and argued the other way, because nothing piled up at review. The quality cost that would have to exist was the part its authors could not show. Vaccari describes the mechanism by which it would go unseen. Twenty-six interviews in one organisation is not a measurement, and the fix he proposes has not been run at scale.
How we Assure
The NCSC has opened a Cyber Adversary Simulation scheme. It assesses commercial providers against a standard the NCSC writes, so that critical organisations and the bodies that oversee them can buy adversary simulation from somebody vouched for. A provider does reconnaissance on the customer’s own technology, exposure and threats. It builds an attack plan from that. Then it works to agreed objectives rather than from procured threat intelligence or a documented list of tactics and techniques. Chris Ensor, a deputy director, put the point as moving beyond traditional penetration testing and the auditing of controls. The scheme starts as a minimum viable product while the NCSC tests demand.
Our position is that the rules an agent runs under have to be executable, because a rule held in a document is a rule nobody can check.
The rules the agents run under are codified as policies with tests rather than held in a standard nobody can check.
A national technical authority has now built its assurance around testing rather than around reading the controls, and it says so in those terms. GitHub reached the same statement in September with a proof-of-presence check, so this is the second source on it. The scheme assures the supplier and not the customer’s defences. No provider count, no pricing and no results are published. The methodology document is not public. A minimum viable product is the NCSC’s own description of how much of this has been tried.