California Gov. Gavin Newsom on Friday issued an executive order to establish a panel of experts tasked with developing safety regulations for artificial intelligence, including the potential creation of an emergency shutdown mechanism for AI systems.
The order directs the panel to explore proposals such as independent third-party safety plans and mandatory emergency shutoff capabilities for AI programs. Newsom’s announcement follows recent public warnings from AI researchers about the technology’s risks, including the possibility of catastrophic outcomes.
Governor’s Rationale and Urgency
Newsom framed the move as a response to what he described as the federal government’s failure to implement meaningful oversight. In a statement, he emphasized the need for "substantial and responsible AI oversight" to address potential dangers before they escalate. The governor’s office noted the panel will provide guidance to strengthen state laws around AI security.
The executive order comes amid heightened scrutiny of AI safety, including the resignation of a researcher from AI company Anthropic, who warned in a social media post that AI development could pose existential threats. The researcher, Jacob Coxon, stated in the post that individuals building AI "earnestly believe that it could kill us all by the end of the decade."
State and Federal Response
State Sen. Chris Cabaldon (D-West Sacramento), chair of the state committee overseeing technology issues, announced plans to hold hearings on AI regulation this fall. Cabaldon criticized existing state rules as insufficient, stating, "We regulate your barber or your sandwich shop more than we regulate [a company] telling us that they may be ending humanity."
At the federal level, Congress has not taken legislative action on AI oversight, and President Donald Trump has resisted calls for regulation. Newsom’s order signals California’s intent to act independently in addressing AI risks.
Previous Stance and Future Implications
Two years ago, Newsom vetoed a state bill that would have mandated a kill switch for AI programs. His latest executive order suggests a shift in approach, prioritizing expert-led recommendations over immediate legislative mandates. The panel’s findings are expected to inform future state policies on AI safety.
The order reflects broader national and international debates over AI regulation, with some lawmakers and industry leaders advocating for stricter controls while others emphasize innovation and economic benefits.