California Governor vetoes SB 1047

Background

Senate Bill 1047 was a California bill that would have required developers of large AI models, and those providing the computing power to train them, to put safeguards and policies in place to prevent catastrophic harm ^[1]. It would also have established a Board of Frontier Models ^[1]. The bill had two main parts: duties on the people building and training large models, and a new public body meant to sit alongside those duties. According to NPR, the bill would have made companies legally liable for harms caused by AI models and required a “kill switch” ^[3]. The bill had been passed overwhelmingly by the California legislature before it reached the governor ^[3]. That set up a final decision by the governor: sign it, veto it, or return it without a signature.

What happened

On 29 September 2024, Governor Gavin Newsom returned Senate Bill 1047 without his signature ^[1]. In his veto message, Newsom said the bill focused only on the most expensive and large-scale models ^[1]. He wrote that this could give the public a false sense of security about controlling the technology ^[1]. He also wrote that smaller, specialised models may emerge as equally or even more dangerous than the models targeted by SB 1047 ^[1]. The message therefore did not treat model size as a reliable proxy for risk. Instead, it pointed to a gap: a law aimed at the largest models might miss dangers that arise elsewhere. The objection was about coverage: which models and deployments a state law should cover.

Key details

Newsom wrote that SB 1047 did not take into account whether an AI system is deployed in high-risk environments, or whether it involves critical decision-making or sensitive data ^[1]. He also said California is home to 32 of the world's 50 leading AI companies ^[1]. That figure was part of his argument about the state's stake in AI policy: California hosts a large share of the industry the bill would have regulated. The bill's author, Senator Scott Wiener, said the veto was a setback for oversight ^[2]. He said voluntary commitments from industry are not enforceable ^[2]. According to NPR, Geoffrey Hinton and Yoshua Bengio signed a letter urging Newsom to sign the bill ^[3]. The objections therefore concerned both the bill's threshold and its context: size and cost on one side, deployment and data on the other.

Impact

Because Newsom returned the bill without his signature, SB 1047 did not become law. The safeguards and the Board of Frontier Models it proposed were not established. The veto left the underlying dispute unresolved: whether California should regulate advanced AI by targeting the largest and most expensive models, as the bill did, or by considering deployment context, critical decision-making, and sensitive data, as Newsom argued ^[1]. Wiener's response framed the choice as one between binding oversight and voluntary industry commitments ^[2]. The letter from Hinton and Bengio showed that the bill had support from two named signatories even as the governor rejected its scope ^[3]. The bill had passed overwhelmingly in the California legislature, but the veto stopped it from taking effect ^[3]. For the bill's supporters, the veto removed a framework that would have made companies legally liable and required a kill switch ^[3]. For the governor, the bill's scope was too narrow to match the range of risks he described ^[1].