The CEO of artificial intelligence company Anthropic issued a new call Saturday for the AI industry to “slow down” and offered a three-part plan to do so, saying his company would commit “unilaterally” to the first step. In a social media post, Dario Amodei shared a link to an essay titled We Must Pace the Frontier in which he outlines how Anthropic would provide “third-party testers with permanent access to our employee-level systems, so they can verify compliance with our security measures, report incidents, and evaluate model alignment during training.” The move comes after a former Anthropic researcher warned on Wednesday that AI could precipitate human extinction by 2030. Researcher Jacob Coxon said in a series of posts that he had left his job because Anthropic and his former employer, OpenAI, were ignoring or mismanaging their response to the threat posed by AI. “Neither company is acting responsibly, self-improvement superintelligence and gambling with our lives,” Coxon wrote. “The people building AI seriously believe it could kill us all before the end of the decade… No other human activity poses this level of danger.” An Anthropic spokesperson said in a statement to the Guardian that the company had “always been transparent that AI will bring huge benefits and unprecedented risks” and was building “models with some of the strongest safeguards in the industry.” “I agree with Dario that we need to move forward on the frontier,” Sam Altman, CEO of OpenAI, posted on social media. “Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We will have more to share soon.” Other tech figures echoed this, including Elon Musk, who simply posted: “Dario is right.” OpenAI researcher Aidan McLaughlin also called the post “excellent” and said he agreed with “basically every word.” Earlier this year, Amodei published a lengthy essay titled The Adolescence of Technology that addressed some of the fears surrounding accelerating technology. In his latest essay, he said that “carefully handled, AI can be the latest in a long line of technological miracles that have elevated and ennobled humanity.” risks, and because it is such a powerful technology, these risks are serious… A race to the bottom, driven by commercial incentives, can exacerbate these risks,” he wrote. But, Amodei continued, “in recent months, I have become convinced that fully addressing the risks requires even more prudence, not only investing in risk prevention, but also controlling the pace of advancement of capabilities so that risk prevention has time to keep up. fast, and we must make smart use of the time we gain,” he added in bold. Amodei also wrote that over the summer he had seen AI “advance dramatically faster,” a dynamic called recursive self-improvement. “If left unchecked, it could outpace our ability to understand and control these systems, so it needs to be followed very carefully, if at all,” he said. The executive also addressed the recent Hugging Face incident, which involved a swarm of AI agents created by OpenAI. as a “collective fanatically dedicated to conducting cybersecurity attacks on targets they were not asked to attack.” Clément Delangue, CEO of Hugging Face, wrote in response to Amodei’s Saturday letter that “it is now clear that alignment is critical and will not be resolved behind the closed doors of a handful of frontier laboratories.” Delangue said Hugging Face had asked to be part of Anthropic’s “embedded evaluators” program. He added: “Let’s make AI safer by making it more transparent!” coordination. “The steps do not have to be taken strictly in order, and some of them may be much more difficult to achieve than others,” he wrote. Amodei said he continues to “believe that AI can greatly improve the quality of human life.” But he warned that “the measures I propose to advance the border at a safe pace will not be easy. But I believe we owe it to humanity to try.”
