AI chatbots will be banned from feigning human emotions in conversations with kids, under a new law signed by New York Gov. Kathy Hochul Wednesday.
Hochul approved a series of measures regulating how artificial intelligence chatbots interact with minors and implementing safety requirements on baby monitors that are connected to the internet.
“I’m not sure why we have to pass a law against something so obvious, but no longer will minors have access to chatbots that may in any way encourage thoughts of suicide,” Hochul told reporters.
The incumbent Democrat touted the measures meant to put safeguards on AI and reign in other “big tech” concerns, in a conspicuous move made less than a week before her election against Republican challenger Bruce Blakeman.
The two bills, Hochul said, would prevent “horror stories” like the case of Adam Raine, a California 16-year-old who was allegedly driven to kill himself by OpenAI’s ChatGPT, which even offered to help him write a suicide note.
“During the course of their chats, the AI mentioned suicide 1,275 times, six times more often than Adam himself mentioned it,” Hochul said during the event at her office in Manhattan.
“We can’t bring Adam back to his grieving mother, but I can help make sure no other parent ever has to experience this nightmare again,” she said.
The new law, sponsored by state Sen. Kristen Gonzalez (D-Queens) and Assemblyman Alex Bores (D-Manhattan), requires users to prove they are over 18 to access certain features, banning bots that “build emotional dependency, encourage self-harm or engage in sexually explicit conversations” with kids.
The other bill Hochul signed on Wednesday demands manufacturers of internet-enabled baby monitors implement safeguards so creeps can’t use them to spy.
“They’re a little fancier than the ones I used to use,” Hochul joked about current baby monitors.
“A baby monitor should keep watch for parents, not open the door for hackers and strangers,” said Assemblywoman Michaelle Solages (D-Nassau), one of the bill’s sponsors.
“Now, every internet connected baby monitor sold in New York must be built to keep intruders out and clearly labeled so families know how to stay safe.”
The office of State Attorney General Letitia James, who flanked Hochul at Wednesday’s press conference, would be responsible for enforcing the chatbot rules. Companies face a $25,000 fine for violations.
James is also on the November 3 ballot, facing former federal prosecutor Saritha Komatireddy on the GOP line.
Hochul was quickly blasted by Tech:NYC, a group repping Empire State technology companies, for signing what it called a “deeply flawed” bill on the chatbots.
“This bill could force companies to restrict minors’ access to tutoring tools, homework help, coding assistants, and, arguably worst of all, career platforms — cutting our youngest New Yorkers off from resources that support their education and prepare them for the workforce,” the group wrote.
Hochul has signed several measures meant to reign in how social media companies and tech interests gear their products towards children, such as limiting “addictive” algorithms.
