California governor blocks landmark AI safety bill

The governor of California, Gavin Newsom, has vetoed a controversial AI safety bill, claiming it could stifle innovation and prompt AI developers to move out of the state.

The proposed bill – the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act (officially known as SB 1047) would have targeted companies developing generative AI – which can respond to prompts with fully formed text, images or audio, as well as run repetitive tasks with minimal intervention.

As one of the first pieces of law focusing on AI safety in the US. It would have required the most advanced AI models to undergo safety testing.

Without the legislation, senator Scott Wiener, who wrote the bill, said companies will be able to continue developing an “extremely powerful technology” without any government oversight.

He claimed Newsom’s block was “a setback for everyone who believes in oversight of massive corporations,” adding that “we are all less safe as a result.”

The bill wanted to make it mandetory for developers building models that cost over $100m to build in a “kill switch” – in other words, the ability to turn an AI system off if it became a threat – and also for AI tools to be tested by a third-party to ensure they minimised ‘grave risk’.

Additionally, the bill would have created whistleblower protections for employees at AI companies who wanted to share safety concerns. If  the legislation had been passed, companies not in compliance with bill could have been sued by the California attorney general.

The bill “does not take into account whether an AI system is deployed in high-risk environments, involves critical decision-making or the use of sensitive data,” argued Newsom.

“Instead, the bill applies stringent standards to even the most basic functions – so long as a large system deploys it,” he added.

Big tech firms have also previously opposed the bill. Alphabet’s Google and Meta expressed their concerns in letters to Wiener. Meta said that the bill threatened to make the state “unfavourable” to AI development and deployment.

Chat GPT creator OpenAI also warned that the law would threaten AI’s growth, and that SB 1047 should be regulated by federal government rather than nationally.

Californians can now request removal of personal data