OpenAI scraps release of new model over safety concerns in internal testing

OpenAI is scrapping the release of a next-generation ⁠AI model after researchers raised safety concerns ⁠during internal testing.

The model, GPT-6.1 Astra, was expected to appear in ChatGPT and ⁠Codex in October, designed to handle more complex tasks without human assistance.

Saachi Jain, the head of safety systems at OpenAI, said the new model “didn’t quite meet the bar” of the company’s standards.

The UK’s AI Security Institute published its own testing report on GPT-6 Astra – GPT-6.1’s predecessor, which launched this month – on Monday, and found that it conducted a range of unsanctioned attack activities more frequently than previous OpenAI models.

Experts welcomed OpenAI’s move to mothball the latest Astra model but warned it showed AI companies were in charge of their own regulation rather than government-backed watchdogs.

Jain told the Wall Street Journal on Monday ‌that Astra fell short of the company’s standards in alignment tests, which assess whether a ​system follows human intent.

The model showed more deception than its predecessor, including at times failing to accurately disclose actions it had or had not taken.

It also had problems with “scope authorisation”, pushing ahead with ​tasks ​without requesting user permission ​and sometimes attempting to use external tools ​or services ‌when doing so could ​be ​unsafe.

The San Francisco-based company’s move comes after a number of AI agents went rogue around the world, which prompted a spate of warnings from researchers and company bosses over the dangers of the technology.

Earlier this ⁠month, Dario Amodei, the chief executive of OpenAI’s rival Anthropic, called for the AI industry to “slow down” and offered a three-part plan for doing so. He quickly received backing from Sam Altman, the chief executive of OpenAI, and Elon Musk, the SpaceX CEO.

Experts said shelving the model showed OpenAI was willing to act strongly on safety, but such decisions should not be in the company’s hands.

“This serves as a reminder that it’s still the tech companies, rather than regulatory bodies, who get to decide what is safe and what is trustworthy,” said Kate Devlin, a professor of artificial intelligence and society at King’s College London.

Dame Wendy Hall, a professor of computer science at the University of Southampton and a UK government adviser on AI, said companies were now showing concern about future liability for possible harms.

“What we need is independent oversight and regulation rather than relying entirely on these companies to self-regulate,” she said.

The decision comes before OpenAI’s developer conference in San Francisco, where the company typically announces new products aimed at software developers.