ANDREW CABALLERO-REYNOLDS/AFP via Getty Images
New York City Council Speaker Julie Menin unveiled a sweeping package of bills on Friday aimed at regulating the AI systems sold and deployed across the city, positioning New York to become one of the first major U.S. cities to impose binding rules on the industry.
“We obviously are not in any way looking to stifle innovation. We want to ensure that New York stays the AI capital of the world, but with that comes responsibility,” Menin said in announcing the legislative push.
The centerpiece measure would require AI systems to undergo outside validation covering data quality, bias, privacy and security before deployment, paired with a mandatory human override, effectively a kill switch, that would let operators shut a system down if it malfunctions or causes harm.
A separate bill would create whistleblower protections that give employees a cut of any fines recovered from companies found in violation, a mechanism the council describes as potentially the first of its kind nationally. Another measure would let New Yorkers sue AI companies when a third party jailbreaks safety controls and that circumvention leads to foreseeable harm, shifting some liability for misuse back onto the companies whose guardrails were bypassed.
The package also includes requirements for AI operators to report safety incidents within 24 hours, a prohibition on companies making false safety claims about their systems, new privacy rules specifically for chatbots, and provisions aimed at defending city systems against algorithmic attacks.
To build the case for the bills, the council has invited the chief executives of the industry’s biggest players, Dario Amodei of Anthropic, Sam Altman of OpenAI, Sundar Pichai of Google, Elon Musk of xAI, and Mark Zuckerberg of Meta, to testify at a hearing scheduled for October 5. None are currently expected to attend, though the council has left open the possibility of issuing subpoenas if they decline.
The stakes are especially local: several of the companies being asked to testify have deepened their physical presence in New York in recent months, including Anthropic’s lease of a 16-story building in the city, underscoring how much the industry itself now has riding on how the city chooses to regulate it.
