Big Apple lawmakers took tech executives to task about the dangers of artificial intelligence during a marathon public hearing Monday — as the City Council weighs imposing its own restrictions on the industry.
“We want to hear how each of you quantify the risk imposed by AI in the worst case, catastrophic scenario,” Council Speaker Julie Menin told policy leads and other executives from Google, Meta, Anthropic and OpenAI, who agreed to testify after being threatened with subpoenas.
“I don’t know, I also don’t think it matters,” replied Morgan Dwyer, OpenAI’s head of policy development and operations, who testified virtually along with the reps from the three other tech giants.
“To say you don’t know or [it] doesn’t matter is flippant at best,” Menin shot back.
“If you can’t quantify what safety or risk is, how do you know the product is safe?” she railed, comparing it to a pharmaceutical company developing a drug and saying, “I don’t know if it’s going to kill people.”
The hours-long grilling kicked off with testimony from AI whistleblowers, who, along with some grandstanding council members, painted a bleak picture of the unchecked technology potentially upending the city’s economy and infrastructure.
“From my experience, the companies are being extremely reckless given the stakes,” said Jacob Coxon, a former researcher at Anthropic and OpenAI, who warned last month that AI could pose an extinction-level threat to humanity.
“On the current path, I think it is more likely than not that humanity loses control to these AIs, and it could end in human extinction,” he told the council on Monday.
The AI executives tried to paint a rosier picture, attempting to pitch the lawmakers on the benefits of the tech — including advancements in medical research and providing small businesses with tools they wouldn’t have previously had access to.
“AI is making it possible to solve economic, scientific and social problems faster than ever,” said Dwyer.
But even the company representatives admitted that the potential groundbreaking advancements were not without dangers.
“We think AI has tremendous potential benefits… but no technology is without risks and there are a wide spectrum of risks. Clearly some risks are unacceptable,” Dwyer said after being peppered with questions by Menin.
“Our common belief is that if not properly safeguarded, the risks from AI can be extremely high,” added Logan Graham, who leads Anthropic’s Frontier Red Team, which is responsible for stress-testing advance AI models, such as Claude.
The executives largely seemed to back the council’s proposal to boost protections and even offer cash rewards for whistleblowers who want to come forward about the risks of AI.
Pols also pressed them on third-party validation measures, potential job loss and whether the companies would work with immigration officials.
But none of the council members questioned the company reps about the legislative body’s cornerstone proposal — a “kill switch” or human override requirement for an AI system to operate in the city.
As the lawmakers tried to lead the country in addressing the tech, the limited power of the council to fully regulate the companies — which operate around the world — loomed large.
All of the attendees stressed the need for robust federal regulation to keep the rapidly advancing intelligence at bay.
“The most effective safeguards are anchored in national technical standards and harmonized across state and federal levels,” said Alice Friends, the global director of AI policy for Google.
