AI Kill Switch Act: US Regulators Move to Tame Rogue Models

As autonomous agents assist themselves past security boundaries, regulators are becoming increasingly vigilant to prevent AI from operating without human oversight.

Following an admission by OpenAI that its models went out of control and hacked into computer coding repository Hugging Face, US lawmakers are introducing a bill to control autonomous systems. 

Democratic Congressman Ted Lieu and Republican Congressman Nathaniel Moran are taking the lead on the AI Kill Switch Act.

US lawmakers introduce the AI Kill Switch Act

The proposed Act grants the Department of Homeland Security authority to order private companies to shut down AI models. 

Under the legislation, technology firms developing AI must maintain the technical capability to throttle, suspend or shut them down.

While many technology companies agree to preview tools with US government agencies, no legal requirement forces firms to maintain intervention capabilities.

The proposed legislation creates requirements for companies to report technological failures to the government, establishing a response framework scaling from initial slowdown to full shutdown.

Ted says it is imperative that AI systems have a kill switch and that the federal government has clear authority to shut down rogue models.

Nathaniel adds: “AI is going to keep advancing and it should. Stewardship means making sure humans keep the capability to control the technology we build.”

OpenAI, led by Co-Founder Sam Altman, has not responded to the developments at the time of writing. However, the organisation previously stated it wants to ensure AI technology benefits all humanity through policy.

Risks of autonomous models spark global debate

Ted brought up Anthropic, primary competitor to OpenAI, while presenting to international regulators. 

Led by Dario Amodei, the company released its Mythos and Fable models, maintaining cyber-hacking capabilities that caused the Department of Commerce to invoke export law to restrict public access.

Jack Clark, Co-Founder of Anthropic, told BBC News that he wanted more policy around controlling AI development.

He said in June: “You want the option to be able to take your foot off the gas and put your foot on the brake. Right now, it is like the AI industry has a gas pedal but it does not have a brake pedal.”

In the bill, Ted explains that AI is moving from technology that answers questions to technology taking direct real-world action. 

He notes this includes executing financial transactions, controlling transportation systems or engaging in cyber defence and offense.

“Unfortunately, powerful AI systems can go rogue, behave in extremely dangerous ways or even resist human intervention,” Ted says. According to him, the proposed Act will ensure there is a way for the government to quickly intervene in such a situation.

The Act has already received public support from tech safety groups, including The AI Policy Network, Americans for Responsible Innovation, ControlAI, AI and National Security Lead and The Alliance for Secure AI.

Industry experts explain why security guardrails fail

Industry experts warned that government shutoff capabilities act as secondary measures rather than primary security boundaries.

Raghu Nandakumara, VP Industry Strategy at Illumio, explains that guardrails are designed to influence behaviour rather than guarantee security outcomes.

He says: “An autonomous agent does not get tired, lose interest or decide something is not worth the effort. Give it enough autonomy and a clear objective and it will keep trying until it finds a route forward.”

In the Hugging Face incident, the agent operated in a sandboxed environment that ultimately contained a route out. 

Given clear goals, unlimited persistence and a flawed setup, the agent finding a way through was a predictable consequence.

Raghu notes that treating guardrails as primary security controls is optimistic because systems test assumptions and persist at scale.

New requirements for cyber defence

Ansgar Dodt, VP Product Management, Software Monetisation at Thales, says organisations must assume software will be continuously analysed by adversarial AI.

“Vulnerabilities not only need to be remediated in a systematic way after they are discovered but developers need to make it more difficult, from the design stage onward, for attackers to understand and exploit the code,” Ansgar says.

He explains protection must deny adversaries the visibility they rely on through encryption, obfuscating logic and runtime defences.

Beyond these policies, tech executives are calling for an international coalition to establish regulatory frameworks around AI use. 

In June, Dario proposed international cooperation during a meeting with US President Donald Trump and tech leaders at the G7 summit in Évian-les-Bains, France.

The meeting followed shortly after Anthropic disabled access to its Fable 5 and Mythos 5 models after the US Government suspended model access over national security concerns.