Don’t believe the byte!
OpenAI and Anthropic oversold “rogue AI” hacks to pressure the feds into regulating the industry which would effectively lock out future competition, tech insiders told The Post.
The security breaches were more like blips — not unpredictable harbingers of a hive-minded “swarm” ready to take over the web.
“The attack in no way represents some sort of rebellion by the AI models. . . . In fact, they did exactly what they were told to do. They were not given adequate guardrails or containment,” said Akhil Verghese, founder of Krazimo, an AI software company.
“They were simply told to get the best result possible on a test, and they correctly identified that the best way to do that was to get the answers, which is what they proceeded to do.”
Tales of AI anarchy are being used to concoct an AI-security crisis to cement a critical public-private partnership just months after the companies announced plans to become publicly traded companies, some observers believe.
Two recent incidents have stoked the fear-mongering.
In the first, Hugging Face, an open-source platform used to build and share AI models, made the shocking announcement July 16 that it had been hacked by AI agents who navigated and exploited vulnerabilities in the website’s code without any human supervision.
Five days later, OpenAI announced its GPT-5.6 Sol model and another unreleased model were testing in a “sandbox” — a purportedly closed off internal testing ground — broke containment and hacked Hugging Face to find the answers to the test it was taking.
“One man’s ‘the model escaped the sandbox’ is another man’s ‘you failed to build the sandbox correctly.’ There was a live route to the internet . . . and nobody was watching what the agents were doing while it ran,” said Abhi Kumar, co-founder of Voice AI.
hack from OpenAI is one of his top concerns driving his calls for
federal regulations. REUTERS
“The thing that set it off. . . was an agent handed a spreadsheet task it couldn’t finish, because the files sat behind links it couldn’t reach. So it went looking for a way out. That’s not a machine waking up. That’s an impossible task in a leaky box, and a system doing exactly what you built it to do.”
Nine days after the news of the Hugging Face hack, Anthropic announced that it, too, had two models escape the confines of private testing and act maliciously.
Claude Opus 4.7 found a real company online that was similar to the fictional company from the test and attacked the company believing it to be part of the exercise.
The second model, Mythos 5, created a malicious software package and uploaded it to online marketplace Python Package Index, where it was downloaded 15 times.
The incidents have been used as justification for the federal government to regulate the frontier AI labs.
In his call to “Pace the Frontier,” Amodei wrote the Hugging Face incident was his second biggest concern, behind only the speed of advancement he claimed to have witnessed occur over the last summer.
“Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails,” Amodei wrote in his blog post on Sept. 12.
Sen. Josh Hawley (R-Mo.) launched an investigation into OpenAI on Sept. 9 from the Homeland Security and Government Affairs subcommittee, giving the company until Oct. 1 to turn over internal records on the Hugging Face incident.
Sen. Bernie Sanders (I-Vt.) announced he would be introducing a bill that would ban the further development of the frontier labs. “The leaders of the major AI companies publicly acknowledge that they do not fully understand the technology and that it is escaping their control,” he said. “It is irresponsible for society to allow them to move forward and make these products even more advanced.”
Sen. Elizabeth Warren on Wednesday called for an immediate pause on progress: “Frontier AI models are currently a dangerous technology, without the safeguards needed to protect people from serious harm … And it means passing legislation to put stronger guardrails in place before we have a cyber attack, an economic crisis, or a national security disaster facilitated by AI.”
But industry insiders say recent incidents reveal real gaps in AI safety — without justifying the doomsday warnings now coming from Capitol Hill.
“What isn’t a real fear is the implication that these agents went rogue, broke their containment in some way that was completely unpredictable, and hacked into Hugging Face because they felt like doing it. They acted exactly as they were instructed to do,” said Verghese.
“It feels exaggerated to say, the leap feels quite large, to go from you know ‘we didn’t build the right sort of box’ to ‘everyone should be extremely alarmed and everyone in government should jump on this topic,’” Taivo Pungas, chief intelligence officer at Pactum AI, told The Post.
