NEW YORK, N.Y. –  New York State Senator Kristen Gonzalez (SD-59), Chair of the Senate Committee on Internet and Technology, introduced a bill prohibiting unsafe chatbot features for minors (S9051). This is a new piece of legislation to prohibit AI chatbots from offering their services to minors when the technology contains certain unsafe features. 

Senator Gonzalez, working closely with the Office of New York State Attorney General Letitia James and Common Sense Media, developed the legislation with the most current research on AI companions and their impacts on young users. The bill aims to protect children from the risks of AI companion chatbots, which have led to tragic outcomes. 

The prohibition would apply to many of the ways chatbots can stimulate emotional connections with minors, targeting various unsafe features including: 

• Outputs suggesting that the chatbot is a real or fictional character, or has a personal or professional connection/relationship role with the user
• Outputs suggesting that the chatbot has personal opinions, pronouns, and experiences human emotions
• Storing and use of data on a user’s mental or physical well-being, acquired from previous uses of the chatbot
• Any engagement or interactions of a sexually explicit nature
• Outputs that endorse harm to self or others, or that encourage secrecy, harm, or sexual content

Companies and chatbot developers in violation of this ban could be subject to both civil action by the affected user, as well as state Attorney General proceedings. The Attorney General of New York would maintain a website to receive and track complaints on these violations. 

Exceptions to this ban include: chatbots whose operators can prove they are not being used by minors (however this determination cannot be made with the requirement of a user sharing government-issued IDs) as well as chatbots that are used internally or for customer service needs. 

Senator Kristen Gonzalez said, “Companion chatbots are endangering our youth. The longer we wait to regulate these applications, the more kids are left vulnerable to exploitation. We must protect children from the risks associated with these chatbots. I am proud to be partnering with Attorney General James to keep kids safe from dangerous uses of AI chatbots, and I urge the legislature to join me in passing these protections.”

New York State Attorney General Letitia James said, “I am proud to support Senator Gonzalez’s bill to protect New York’s kids from the harms of chatbots. There is already overwhelming evidence that too many of our children are being exposed to inappropriate and dangerous content while using chatbots, and this bill provides critical safety protections against those harms as the AI landscape continues developing in New York.”

Common Sense Media Founder and CEO James P. Steyer said, “Common Sense Media applauds State Senator Kristen Gonzalez and Attorney General Letitia James for their leadership on this much-needed legislation to protect kids from some of the particular risks artificial intelligence poses to young people. In our increasingly AI-powered world, parents need someone to look out for the interests of kids and families. Three in four teens use AI companions, but the research is clear that no one under 18 should use unsafe AI companion chatbots. Through strong, privacy-protective age assurance measures and clear feature-based  guardrails on chatbots, this legislation will help to keep unsafe AI companion chatbots away from kids in New York.  We look forward to working with Empire State leaders this session to score another big win for kids' online safety in New York.”

Rahnold Thomas, Technology and Innovation Chair, NAACP New York State Conference said, “AI chatbots can be powerful learning supports by helping students explore ideas, practice skills, and access resources when a teacher or parent isn’t immediately available. But as these tools become more autonomous and agentic, it’s critical that they operate with clear boundaries and age-appropriate safeguards. Without those guardrails, we risk creating dependency, misinformation, or harm in high-risk situations. The NAACP New York State Conference supports this legislation as a practical, common-sense step toward maximizing the benefits of AI while ensuring accountability, safety, and responsible use. Thoughtful regulation is the only way to ensure transparency and accountability are prioritized as we adopt and scale innovation.”

National Alliance on Mental Illness said, "Because most mental health conditions begin in childhood, adolescence, or young adulthood, NAMI New York State strongly advocates for robust youth mental health services and protections. Without the right safeguards, AI can put young people at risk. AI tools are not a substitute for mental health care, but many people are already using them to ask questions or find support. Some tools can help when used responsibly, but others can cause harm. Adolescents are particularly vulnerable to these risks given brain development and identity exploration. Parents and youth deserve clear, trustworthy information, and knowing AI tools will not harm their mental health.”

Sam Hiner, Co-Founder and Executive Director, Young People's Alliance said, “Young people do not want their friends and communities to be sucked into addictive chatbots. I’m glad to see the continued leadership of New York on this issue as Senator Gonzalez introduced this legislation to fight back against Big Tech’s disruption of the community for the sake of profit and increased engagement.”  

Senator Gonzalez sponsors a series of legislation regulating AI chatbots, including:

• AI Licensed Professionals (S7263) - Imposes liability for damages caused by a chatbot impersonating certain licensed professionals
• Chatbot Disclosure (S934A) - Requires the owner, licensee or operator of a generative artificial intelligence system to conspicuously display a warning on the system's user interface that is reasonably calculated to consistently apprise the user that the outputs of the generative artificial intelligence system may be inaccurate and/or inappropriate.

Chatbot Penalties (S5668) - Imposes liability for misleading, incorrect, contradictory or harmful information to a user by a chatbot that results in financial loss or other demonstrable harm.