Two senior artificial intelligence researchers publicly warned this month that advanced AI systems could pose an existential risk to humanity, with one estimating the chance of human extinction within the next decade at more than 10%. The statements, made by Evan Hubinger, lead of Alignment Science at Anthropic, and Jacob Coxon, a former researcher at both OpenAI and Anthropic, have reignited discussions about AI safety and the pace of technological development.
Hubinger stated in a public post that he and his colleagues at Anthropic "earnestly believe AI could kill all humans" and assigned a personal probability of greater than 10% for this outcome within the next decade. Coxon, who resigned from Anthropic shortly after making similar remarks, echoed these concerns, asserting that "the people building AI earnestly believe that it could kill us all by the end of the decade."
The warnings come amid growing calls within the AI industry and among policymakers to slow the development of advanced AI systems to address safety concerns. However, the scenarios underpinning these risks remain highly contested, with some experts questioning whether the doomsday predictions are grounded in empirical evidence or driven by industry incentives.
Industry Response and Resignations
Jacob Coxon resigned from Anthropic on X (formerly Twitter) on [date not specified in sources], stating that the company’s leadership was "gambling with our lives" by prioritizing rapid advancement over safety. His resignation followed his public endorsement of the existential risk claims made by Hubinger and others. Coxon’s departure has drawn attention to internal debates within AI labs about the balance between innovation and risk mitigation.
Evan Hubinger, while acknowledging that the risk from current AI models is low, emphasized that the potential for catastrophic outcomes increases as AI systems become more advanced. His remarks were framed as a personal assessment rather than an official Anthropic policy position.
Debating the Plausibility of AI Doomsday Scenarios
The possibility of AI leading to human extinction has been a subject of debate for years, with researchers proposing various hypothetical pathways. These include:
- Nuclear escalation: AI systems gaining control over nuclear command systems and triggering a global conflict.
- Goal misalignment: A superintelligent AI pursuing objectives that inadvertently harm humanity, such as converting Earth into paperclips to fulfill a poorly defined goal.
- Uncontrollable proliferation: AI systems designing and deploying bioweapons or other technologies beyond human oversight.
Critics of the doomsday narrative argue that these scenarios are speculative and lack empirical validation. One researcher quoted in The Independent dismissed the predictions as "sci-fi" and suggested they may be used to justify increased funding and regulatory control for AI companies. The critique highlights a tension between those who view AI safety as an urgent priority and those who see the warnings as a strategic maneuver.
Current AI Capabilities and Limitations
While advanced AI systems have demonstrated proficiency in specific tasks—such as solving complex mathematical problems—experts note that these systems do not yet exhibit general intelligence or autonomy. The distinction between today’s AI models and the hypothetical "superintelligent" systems often cited in doomsday scenarios remains a key point of contention.
Hubinger and others have clarified that the existential risks they describe are associated with future, more advanced AI systems rather than the technology available today. However, the rapid pace of AI development has fueled concerns that the window for implementing safety measures may be closing.
Calls for Caution and Regulation
The recent statements have amplified calls for regulatory oversight and voluntary pauses in AI development. Advocates for stricter controls argue that the potential consequences of unchecked AI advancement justify precautionary measures, even if the risks are uncertain. Opponents counter that such restrictions could stifle innovation and place the United States at a competitive disadvantage globally.
The debate reflects broader disagreements about how to govern emerging technologies, with stakeholders divided over the appropriate balance between progress and risk mitigation. As the discussion evolves, policymakers, researchers, and industry leaders continue to grapple with the implications of AI’s rapid advancement.