On September 8, former Anthropic researcher Jacob Coxon publicly warned that AI development could pose an existential risk to humanity, stating that those building the technology "earnestly believe that it could kill us all by the end of the decade." Within hours, Evan Hubinger, Anthropic’s lead alignment researcher, endorsed Coxon’s remarks, asserting in a social media post that he personally estimates a greater than 10% chance of human extinction from AI within the next decade.
The warnings followed the September 1 release of Anthropic’s Fable and Mythos 5.1 models, which are capable of autonomously working on tasks for extended periods, and OpenAI’s Astra model, released September 3, both of which demonstrate advanced problem-solving capabilities. On September 8, Anthropic CEO Dario Amodei published an essay titled "We Must Pace the Frontier," arguing for a slower development pace to ensure safety. Within a day, Sam Altman (OpenAI), Demis Hassabis (Google DeepMind), and Elon Musk publicly endorsed Amodei’s call for a coordinated slowdown in AI advancement.
Industry leaders frame AI risks as a national security priority
Amodei’s essay, published the same day as Coxon’s resignation, positioned AI safety as a critical issue, stating that the technology’s rapid progress outpaces researchers’ ability to control it. His stance was echoed by Altman, Hassabis, and Musk, who collectively warned of the dangers of unchecked AI development. On September 9, The New York Times led its front page with an analysis by David Sanger, describing AI dominance as the "top national security issue of the century."
Diverging views on regulation and control
While industry leaders advocate for a slower development pace, David Sacks, a technology investor and former Trump administration AI adviser, criticized what he termed "AI doomerism" in a September 9 interview with CBS News. Sacks argued that fears of AI posing an existential threat are overblown and that the current discourse is driven by political incentives rather than technical realities. He stated, "I think this is becoming a panic... We're in an election season, and there is an incentive for political actors to amplify the fear."
Sacks, who serves as co-chair of President Trump’s Council of Advisors on Science and Technology, emphasized that AI companies—not governments—must ensure their products are safe. He added, "If you can't control it, then don't do it." His comments contrast with Amodei’s call for industry-wide coordination, suggesting that self-regulation, rather than external oversight, should guide AI development.
Background: The rise of autonomous AI systems
The latest AI models, including Anthropic’s Fable and Mythos 5.1 and OpenAI’s Astra, represent a shift toward autonomous operation, capable of completing complex tasks independently over extended periods. This advancement has intensified debates about AI’s potential risks, particularly regarding unpredictable behavior, loss of human control, and unintended consequences.
Key stakeholders weigh in
- Anthropic’s Dario Amodei: Called for a slower development pace to address safety concerns, framing AI risks as a national security issue.
- OpenAI’s Sam Altman: Endorsed Amodei’s essay, signaling industry-wide concern over rapid AI advancement.
- Google DeepMind’s Demis Hassabis: Joined the call for a coordinated slowdown, emphasizing the need for caution.
- Elon Musk: Publicly supported the industry leaders’ stance, warning of AI’s potential dangers.
- David Sacks: Argued that fears of AI extinction are exaggerated and that the current discourse is politically motivated.
What’s next?
The industry’s push for a slower development pace faces competing priorities: balancing innovation with safety, addressing geopolitical competition, and determining the role of government oversight. The debate has expanded beyond technical experts to include political leaders, with figures such as Barack Obama, Kamala Harris, and Bernie Sanders publicly weighing in on AI’s risks and regulation.