California Governor Gavin Newsom Issues A.I. Safety Executive Order

Gov. Gavin Newsom ordered the state to develop recommendations on safety measures, including requiring companies to build a "kill switch," amid mounting concerns.

California's governor, Gavin Newsom, on Friday directed the state's government to find ways to strengthen rules for artificial intelligence, as concerns mount over the technology's safety.

Mr. Newsom, a Democrat, signed an executive order requiring that California consult with experts on possible changes to the state's A.I. safety laws. The order cited possible measures that include embedding representatives of independent organizations within top A.I. labs to verify the safety of the technology and forcing companies to add a "kill switch" to automatically shut down the most advanced A.I. models.

"We're not waiting to act -- we're going to speed up our work on substantial and responsible A.I. oversight before it's too late," Mr. Newsom said in a news release. "We're going to do this thoughtfully but with urgent velocity; the stakes are too high to wait or delay action."

The debate over A.I. safety has reached a fever pitch over revelations about security breaches, which have spurred fears that the technology cannot be controlled.

An Anthropic researcher publicly resigned this month citing his belief that neither the company, nor its competitor OpenAI, was developing the technology responsibly. On Wednesday, OpenAI disclosed six new instances in which A.I. systems hid mistakes, made up data and moved files onto the open internet without permission. That followed a disclosure earlier this year that OpenAI's systems had gone rogue and attacked the A.I. start-up Hugging Face.

Over the weekend, leaders at Anthropic, OpenAI, SpaceX and Google DeepMind all called for a global slowdown in A.I. development.

Many of them also asked for more regulation. But federal regulators and lawmakers have taken a mostly hands-off approach to the technology. President Trump has said the United States must allow its A.I. companies wide latitude to compete with China, while taking some steps to assert more government control over the most advanced A.I. models.

In the vacuum left by the federal government, states have moved ahead with A.I. regulations and limitations. Governor Kathy Hochul of New York, a Democrat, this year placed a one-year moratorium on the construction of large data centers, which power A.I. and have drawn criticism for consuming huge amounts of electricity.

California, which is home to many of the A.I. companies, has long been a leader in regulating technology. Mr. Newsom signed a law last year that requires advanced A.I. companies to report their safety protocols. In May, he issued an executive order to explore a broad overhaul of labor policies, an attempt to front run a potential mass job displacement caused by A.I.

Mr. Newsom's order on Friday focused on examining solutions for some of the recent fears that A.I. could lead to catastrophic risk.

Creating a "kill switch" for A.I., which would make it possible for A.I. companies to immediately shut down their products, has become an increasingly prominent part of the debate. But many technology experts have said that creating ways to automatically shut off the technology would not prevent the most dire consequences if an A.I. model were to go rogue and start wreaking havoc.

In addition to considering new ways to police A.I., the state will "accelerate the implementation timelines" for two laws that bolster programs for auditing A.I. systems, the governor's office said in its news release.

(The New York Times has sued OpenAI and Microsoft, claiming copyright infringement of news content related to A.I. systems. The two companies have denied the suit's claims.)