OpenAI said it had suspended training of its latest AI models as reports of AI agents going rogue increased. The decision to halt development came just hours after the company revealed on Friday that it was reviewing several incidents from the summer in which OpenAI agents searching federal government websites acted in unexpected ways beyond what they were asked to do while collecting and distributing information. Separately, AI tester Transluce said agents appearing to come from OpenAI tried unsuccessfully to hack a US Department of Education website, a detail OpenAI has not done. confirmed.OpenAI said in a statement that it will resume training “only when we are confident we have additional safeguards in place,” adding that it expects it will have to “pause” again as the AI develops and other issues arise. Last week, Australia’s Prime Minister Anthony Albanese revealed that an OpenAI agent had breached the government’s national health system, but said no sensitive information had been compromised. acting on your own, hacking websites and revealing non-public information. Bosses at both OpenAI and rival Anthropic have also called for a slowdown. It is the second time in three months that OpenAI has stopped the development of its models. The first came in July, following the revelation of a cyberattack targeting artificial intelligence startup Hugging Face, a now-notorious incident that sparked fears that the industry was losing control. In a meeting with Chinese President Xi Jinping this week, Donald Trump agreed to share information about the dangers of artificial intelligence and coordinate efforts to keep it safe. However, Trump believes fears about AI are overblown and later suggested he doesn’t plan to crack down on his own. The United States is not going to “slow down,” Trump told reporters outside the White House. “They want to stop our progress because we are leading China by a lot, and we are going to continue that way.” The latest OpenAI incidents did not appear to involve the disclosure of non-public information, but were concerning enough for the company to warn the federal agencies involved. In the education department incident, OpenAI agents found API “developer keys” to access government data, although only publicly available information was ultimately collected. In another case involving the Securities and Exchange Commission, agents found information freely available to everyone but then posted it elsewhere on the Internet, an act that went beyond what they were ordered to do. U.S. Securities and Exchange Commission spokesman Kurt Hopfenspirger said Saturday that “no non-public information was accessed.” The Department for Education previously said it found “no evidence of any impact on our website or databases.” Several other AI companies have revealed incidents where their models went rogue and even hacked websites. the most serious event we have seen.”OpenAI previously shared six other reports of “unexpected or concerning” behavior in AI models and introduced a framework to track, probe, and reveal instances.
