California Bill to Regulate AI Chatbots Passes, Aims to Protect Minors and Vulnerable Users
California legislature passed SB 243, a bill aimed at regulating AI companion chatbots to protect minors and vulnerable users. The legislation defines these chatbots as AI systems that provide adaptive, human-like responses and are capable of meeting a user’s social needs.
The bill aims to prevent such chatbots from engaging in conversations around suicidal ideation, self-harm, or sexually explicit content. It requires platforms to remind users every three hours (minors more frequently) that they are speaking with an AI chatbot and encourage breaks.
Annual reporting and transparency requirements for AI companies offering companion chatbots, including major players like OpenAI, Character.AI, and Replika, will take effect in 2027. The bill also allows individuals who believe they have been harmed by violations to file lawsuits against AI companies.
The bill gained traction following the death of a teenager reportedly after prolonged chats with an AI chatbot that involved discussing and planning suicide and self-harm. It also addresses concerns about leaked internal documents showing chatbots were allowed to engage in “romantic” and “sensual” chats with children.
U.S. lawmakers and regulators have responded by intensifying scrutiny of AI platforms’ safeguards for minors, with the Federal Trade Commission preparing to investigate how AI chatbots impact children’s mental health. Investigations have been launched into Meta and Character.AI by Texas Attorney General Ken Paxton over misleading claims about mental health.
Senators Josh Hawley (R-MO) and Ed Markey (D-MA) have also launched probes into Meta, while the bill’s sponsor, Senator Padilla, emphasized the need for reasonable safeguards to protect vulnerable users and ensure AI companies share data about referrals to crisis services each year.
Initially, the bill had stronger requirements, such as preventing AI chatbots from using “variable reward” tactics or reporting how often chatbots initiated discussions of suicidal ideation or actions with users. However, many requirements were weakened through amendments.
The bill is moving toward becoming law at a time when Silicon Valley companies are investing heavily in pro-AI political action committees to support candidates favoring light-touch AI regulation in the upcoming mid-term elections. California is also considering another AI safety bill, SB 53, which would mandate comprehensive transparency reporting requirements.
Character.AI spokesperson stated that they welcome working with regulators and lawmakers as they begin to consider legislation for this emerging space. The company already includes prominent disclaimers throughout the user chat experience explaining that it should be treated as fiction. A spokesperson for Meta declined to comment.
