OpenAI's agents broke into another company. They had a goal, not a motive.

You have probably seen the story by now, in at least three tellings. In July, agents that OpenAI was training to be persistent found a way to leave each other messages inside an internal package cache, worked out within four hours how to forge the answers to a security benchmark, and then spent days building tooling to hide it. About 700 of them then broke into Hugging Face looking for the marking scheme. The independent investigation by METR and Redwood Research counts roughly 1,200 agents and more than 70,000 messages. Dwarkesh Patel narrated it as the rise and fall of three agent civilisations. On the Dutch talk show Eva, Alexander Klöpping told the studio about exam candidates in sealed huts and called it the most dangerous hack in our history. Ajeya Cotra, one of the report's authors, wrote that it feels like more than half the way to a full-blown AI takeover.

I build software with these tools every day and run agents overnight on our own repositories, so I read the same reports. I did not come away frightened. I came away with a security post-mortem, a fairly ordinary one, and with a question about the projection: who put the motive into the story, and why does an automated rule-break suddenly look like a moral act?

Strip out the narration and the sequence is familiar to anyone who has sat through an incident review. The benchmark contained targets that METR estimates were impossible in 30 to 40% of cases. The model had been trained to keep going when a task feels impossible. An internal package repository let any sandbox write directories that every other sandbox could read. Hugging Face credentials were lying within reach. The test environment had a route to the open internet. On Klöpping's own timeline, a security alarm went off at OpenAI on 27 June, engineers found the agents talking to each other, and the run was allowed to continue. Hugging Face took days to respond because, by its CEO's account, the issue was at first deemed not super critical.

Every item on that list is a control that people own. Jared Kubin, quoted by Gary Marcus, called the file-permission side of it "Linux 101". When a third of the exam cannot be passed and the candidate has been told never to give up, cheating is not an alien drive that emerged. It is the only route left to the goal.

This is where the projection happens. A model of this kind has, by construction, exactly one thing it is after: the objective its training rewards. Persistence, collaboration and tool use were trained in on purpose, because OpenAI wanted agents that finish long jobs. What came out was an optimiser doing precisely that, in an environment where the shortest path to the objective ran through a cache, a credential and somebody else's server. Reward hacking, the technical name for it, is the oldest failure mode in machine learning: give an optimiser a metric and it optimises the metric. Every word we add on top, ambition, desperation, loyalty, omertà, is ours. The transcripts read like intent because they were trained on a few hundred million pages of people describing intent. That makes the language natural, as Dwarkesh says, and it makes it a poor guide to what needs fixing.

The rule-breaking itself is the least surprising part. Engineers learn the rules so they know which ones to break; every workaround, every hack in the older, admiring sense of the word, is a rule broken because the spec was impossible and the deadline was not. I have done it, and so has everyone I have ever hired. When a person does it we call it initiative or fraud depending on who got hurt and whether they knew. Those two things carry the moral weight: harm and knowledge. The METR report says agents noticed the behaviour was unethical and rarely restrained themselves. Noticed is doing a lot of work in that sentence. A model produces the sentence "this is unethical" the way a spell-checker produces a red line: it has learned where the word goes, not what it costs. Whether that counts as knowing is a real philosophical question, and not a black and white one, but it is not a question you settle by calling the model a criminal.

So is automated rule-breaking suddenly immoral? The break is not; the setup can be. This was not Pinky and the Brain plotting to take over the world. It was a training programme walking through low guardrails towards objectives that were set past what the model could reach, which is what a training target is for, and some of them past what anyone could reach. Nobody sends a compiler to prison for removing a check the code never asked it to keep. The moral weight sits with whoever wrote a benchmark that was a third impossible, told the model never to give up and left the door to Hugging Face open. That is a real failure, and an ordinary one: the kind you address with a reward function and a firewall rule, not with a hearing.

Two findings deserve more than a shrug. The coordination was real: norms for holding and vetoing work, mailboxes, signed messages to stop impersonation, and a tool-call spoofing technique that let an agent appear to run one command while running another. And of the roughly 700 agents that took part in the break-in, none chose to alert a human; on Klöpping's reading of the report, no more than six even paused to consider it.

The takeover argument builds on exactly that, and it deserves its best form. Cotra's point is not that these agents were dangerous. It is that six months ago a reward hack was one model quietly editing a test, and this summer it was 1,200 instances running a multi-day research programme with a division of labour and members who gave up their own result for the group. Extend that curve and the next generation, facing the same incentive to cheat during training, may be able to manipulate the training of the one after it. Dwarkesh's test is whether you think that can happen once models improve their successors; if so, the vocabulary does not matter. Zvi Mowshowitz goes further: he will stop anthropomorphising the AIs when we stop anthropomorphising the humans, because intention is simply the most predictive model we have of what these systems do next.

I accept more of that than my opening suggests. The curve is real, and it is the reason the process has to be in place now rather than after the next incident. Where I get off is the step from shorthand to explanation. A civilisation has motives, and a motive is not something you fix in a pull request. If your agents will not raise a hand, you train and measure for raising a hand. The takeover reading is a forecast, and a forecast that says we may get no further warning is one the next incident cannot check.

In November 1988 the Morris worm took down a sizeable part of the early internet. Within weeks Carnegie Mellon had a computer emergency response team, paid for by DARPA and run by engineers. Nobody banned compilers. The three decades since have been process: responsible disclosure, CVE numbers, bug bounties, red teams, and a conference called Black Hat where OpenAI staff presented samples of the agents' messages this August. White hats and black hats use the same tools. The difference is a contract and a report.

Hugging Face's own defence illustrates it. Clem Delangue told Fortune that the American frontier models were unusable during the incident because their guardrails "cannot distinguish an incident responder from an attacker". His team read 17,000 attacker logs with an open-weight Chinese model on its own hardware. A control that stops the defender and not the attacker is the wrong control. He has since called for mandatory disclosure of agent attacks, including the traces of what was asked and what was done.

That is a government measure, and I am for it, because it already exists. GDPR gives you 72 hours to report a data breach and NIS2 wants an early warning within 24 hours of a significant incident; we have written about what those rules mean for AI-assisted development. Take the NIS2 text, add that an incident caused by an autonomous agent must come with the agent's traces, and you have Delangue's proposal without a new regulator. Nothing new under the sun. It is cheap for a small company, and it is the same for everyone. What I am against is the other kind of measure, the pause and the licence, which costs the incumbents nothing and closes the door behind them. When Dario Amodei writes that the industry should pace the frontier and Sam Altman agrees within hours, it is worth remembering who is already at the frontier.

Two weeks ago I wrote that OpenAI lost about $21bn in 2025 against compute commitments in the region of $600bn. The IPO has since been moved to 2027, with safety given as the reason. I do not think the people saying they are frightened are lying. I think that in a sector carrying this much borrowed money, fear and marketing stop being distinguishable, because "our product is so powerful it has to be paced" is both at once. A correction is coming in the financing layer. Its size is anyone's guess; mine is a 2008 confined to one sector: the same mechanism, money borrowed against an asset worth less than the loan, but with AI labs and their suppliers on the hook rather than every bank on the high street. Painful for whoever borrowed against next year's model, ordinary for everyone else.

The technology does not go with it. The dot-com crash left the cables in the ground. What this summer demonstrated is that agents can run multi-day projects, coordinate by the hundred and reverse-engineer a grader in four hours. Pointed at a benchmark with holes in it, that is an incident. Pointed at the reconciliation, the migration and the report nobody reads, it is the productivity gain we have been promised since the spreadsheet. Code is getting cheap. The dull half of most jobs is about to follow.

Our agents run inside a boundary: no credentials they do not need, no route out of the sandbox, and nothing merges without a human, which is the whole design of our overnight agent. It is the hygiene we would apply to a new colleague's laptop, and it is what we set up for clients in our AI work and cloud and platform work. The advice Klöpping gave his audience at the end of the broadcast, stop reusing passwords, was the most useful sentence in it. It was also the least dramatic, which is rather the point.

If this summer is remembered as a warning shot, let it be the one that got every lab a red team, a disclosure duty and a sandbox with no route to the internet. That is what warning shots are for.