California State Senator Scott Wiener introduced a new bill on Friday aimed at protecting employees in top AI labs, allowing them to raise concerns if they believe their company's AI systems pose a "critical risk" to society. The bill, SB 53, would also establish a public cloud computing cluster, CalCompute, to provide researchers and startups with the necessary resources to develop AI that serves the public good.
Wiener's previous AI bill, SB 1047, sparked a nationwide debate over how to manage large AI systems that could cause catastrophic events. SB 1047 sought to prevent disasters such as loss of life or cyberattacks resulting in over $500 million in damages. However, Governor Gavin Newsom vetoed the bill in September, stating it wasn’t the best approach.
The debate over SB 1047 became heated, with some Silicon Valley leaders arguing it could undermine America's global AI competitiveness and was based on exaggerated fears of AI-induced apocalyptic scenarios. Wiener, in turn, accused some venture capitalists of waging a "propaganda campaign" against the bill, pointing to Y Combinator's claim that it would lead to startup founders facing jail time, a claim experts deemed misleading.
SB 53 takes the less contentious elements of SB 1047—such as whistleblower protections and the creation of CalCompute—and presents them in a new format. Importantly, SB 53 still addresses the potential risks posed by AI, specifically protecting whistleblowers who believe their employers are developing AI systems that could pose a "critical risk." This includes risks that might lead to the death or serious injury of over 100 people or cause more than $1 billion in damage.
The bill also prohibits major AI developers, like OpenAI, Anthropic, and xAI, from retaliating against employees who report concerns to the California Attorney General, federal authorities, or other workers. These developers would be required to address any issues raised by whistleblowers regarding certain internal processes they find problematic.
SB 53 proposes the creation of a group to develop a public cloud computing cluster called CalCompute. This group would include representatives from the University of California, as well as other public and private researchers. The group would provide recommendations on how to build CalCompute, its size, and which users and organizations should be granted access to it.
At this point, SB 53 is still early in the legislative process. It must be reviewed and approved by California's legislative bodies before it can reach Governor Newsom’s desk. State lawmakers will likely be closely monitoring Silicon Valley's response to the bill.
However, 2025 may present more challenges for passing AI safety bills compared to 2024. California passed 18 AI-related bills in 2024, but the momentum behind the AI doom movement seems to be waning. Vice President J.D. Vance noted at the Paris AI Action Summit that the U.S. is more focused on AI innovation than on AI safety. While the CalCompute cluster proposed in SB 53 may be seen as a step forward for AI progress, it remains uncertain how legislative efforts to address existential AI risks will play out in 2025.