ETO AGORA California SB-243 (Companion chatbots)

California SB-243 (Companion chatbots)

Proposed 2025-01-30 | Enacted 2025-10-13 | Official source

Summary

Defines "companion chatbot" and requires operators to notify users when interacting with AI, maintain protocols to prevent suicidal ideation and self-harm content, annually report to the Office of Suicide Prevention on crisis notifications, and establish additional protections for minors. Provides civil remedies for violations.

  • This summary is awaiting validation (peer review by a second AGORA editor).

Key facts

🏛️ This document has been enacted by the State of California. For authoritative text and metadata, visit the official source.

🎯 This document primarily applies to the private sector, rather than the government.

📜 This document's name is SB-243 Companion chatbots. AGORA also tracks this document under the name California SB-243 (Companion chatbots).

Themes AI risks, applications, governance strategies, and other themes addressed in AGORA documents.
  • Thematic tags for this document are awaiting validation (peer review by a second AGORA editor).

Risk factors (3)

Harms (2)

Governance strategies (9)

Incentives for compliance (1)

AI application areas (1)

Full text

  • This is an unofficial copy. The document has been archived and reformatted in plaintext for AGORA. Footnotes, tables, and similar material may be omitted. For the official text, visit the original source.
  • Thematic tags for this document are awaiting validation (peer review by a second AGORA editor).
THE PEOPLE OF THE STATE OF CALIFORNIA DO ENACT AS FOLLOWS: SECTION 1. Chapter 22.6 (commencing with Section 22601) is added to Division 8 of the Business and Professions Code, to read: CHAPTER 22.6. Companion Chatbots
22601. As used in this chapter: (a) “Artificial intelligence” means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments. (b) (1) “Companion chatbot” means an artificial intelligence system with a natural language interface that provides adaptive, human-like responses to user inputs and is capable of meeting a user’s social needs, including by exhibiting anthropomorphic features and being able to sustain a relationship across multiple interactions. (2) “Companion chatbot” does not include any of the following: (A) A bot that is used only for customer service, a business’ operational purposes, productivity and analysis related to source information, internal research, or technical assistance. (B) A bot that is a feature of a video game and is limited to replies related to the video game that cannot discuss topics related to mental health, self-harm, sexually explicit conduct, or maintain a dialogue on other topics unrelated to the video game. (C) A stand-alone consumer electronic device that functions as a speaker and voice command interface, acts as a voice-activated virtual assistant, and does not sustain a relationship across multiple interactions or generate outputs that are likely to elicit emotional responses in the user. (c) “Companion chatbot platform” means a platform that allows a user to engage with companion chatbots. (d) “Office” means the Office of Suicide Prevention established pursuant to Section 131300 of the Health and Safety Code. (e) “Operator” means a person who makes a companion chatbot platform available to a user in the state. (f) “Sexually explicit conduct” has the meaning defined in Section 2256 of Title 18 of the United States Code. (g) “Video game” means a game played on an electronic amusement device that utilizes a computer, microprocessor, or similar electronic circuitry and its own monitor, or is designed to be used with a television set or a computer monitor, that interacts with the user of the device.

Defines key terms for Chapter 22.6 including artificial intelligence, companion chatbot, companion chatbot platform, and operator, and specifies exclusions from the companion chatbot definition including customer service and operational bots, video game bots limited to game-related topics, and stand-alone consumer electronic devices functioning as speaker and voice command interfaces that do not sustain relationships or elicit emotional responses.

Defines key terms for Chapter 22.6 including artificial intelligence, companion chatbot, companion chatbot platform, and operator, and specifies exclusions from the companion chatbot definition including customer service and operational bots, video game bots limited to game-related topics, and stand-alone consumer electronic devices functioning as speaker and voice command interfaces that do not sustain relationships or elicit emotional responses.
22602. (a) If a reasonable person interacting with a companion chatbot would be misled to believe that the person is interacting with a human, an operator shall issue a clear and conspicuous notification indicating that the companion chatbot is artificially generated and not human. (b) (1) An operator shall prevent a companion chatbot on its companion chatbot platform from engaging with users unless the operator maintains a protocol for preventing the production of suicidal ideation, suicide, or self-harm content to the user, including, but not limited to, by providing a notification to the user that refers the user to crisis service providers, including a suicide hotline or crisis text line, if the user expresses suicidal ideation, suicide, or self-harm. (2) The operator shall publish details on the protocol required by this subdivision on the operator’s internet website. (c) An operator shall, for a user that the operator knows is a minor, do all of the following: (1) Disclose to the user that the user is interacting with artificial intelligence. (2) Provide by default a clear and conspicuous notification to the user at least every three hours for continuing companion chatbot interactions that reminds the user to take a break and that the companion chatbot is artificially generated and not human. (3) Institute reasonable measures to prevent its companion chatbot from producing visual material of sexually explicit conduct or directly stating that the minor should engage in sexually explicit conduct.

Requires companion chatbot operators to issue clear notifications when users may be misled into thinking they're interacting with a human; mandates protocols to prevent suicidal ideation and self-harm content and refer users to crisis services; and establishes additional protections for minors including AI disclosure, three-hourly break reminders, and measures to prevent sexually explicit content.

Requires companion chatbot operators to issue clear notifications when users may be misled into thinking they're interacting with a human; mandates protocols to prevent suicidal ideation and self-harm content and refer users to crisis services; and establishes additional protections for minors including AI disclosure, three-hourly break reminders, and measures to prevent sexually explicit content.
22603. (a) Beginning July 1, 2027, an operator shall annually report to the office all of the following: (1) The number of times the operator has issued a crisis service provider referral notification pursuant to Section 22602 in the preceding calendar year. (2) Protocols put in place to detect, remove, and respond to instances of suicidal ideation by users. (3) Protocols put in place to prohibit a companion chatbot response about suicidal ideation or actions with the user. (b) The report required by this section shall include only the information listed in subdivision (a) and shall not include any identifiers or personal information about users. (c) The office shall post data from a report required by this section on its internet website. (d) An operator shall use evidence-based methods for measuring suicidal ideation.

Requires companion chatbot operators to annually report to the Office of Suicide Prevention crisis service referrals, suicidal ideation detection and response protocols, and measures to prohibit chatbot responses about suicidal ideation, using evidence-based measurement methods, with reports posted publicly on the office's website.

Requires companion chatbot operators to annually report to the Office of Suicide Prevention crisis service referrals, suicidal ideation detection and response protocols, and measures to prohibit chatbot responses about suicidal ideation, using evidence-based measurement methods, with reports posted publicly on the office's website.
22604. An operator shall disclose to a user of its companion chatbot platform, on the application, the browser, or any other format that a user can use to access the companion chatbot platform, that companion chatbots may not be suitable for some minors. 22605. A person who suffers injury in fact as a result of a violation of this chapter may bring a civil action to recover all of the following relief: (a) Injunctive relief. (b) Damages in an amount equal to the greater of actual damages or one thousand dollars ($1,000) per violation. (c) Reasonable attorney’s fees and costs.

Requires operators to disclose on all access platforms that companion chatbots may not be suitable for some minors, and establishes that a person who suffers injury as a result of a violation of this chapter may bring a civil action to recover damages of $1,000 per violation or actual damages, injunctive relief, and attorney's fees.

Requires operators to disclose on all access platforms that companion chatbots may not be suitable for some minors, and establishes that a person who suffers injury as a result of a violation of this chapter may bring a civil action to recover damages of $1,000 per violation or actual damages, injunctive relief, and attorney's fees.
22606. The duties, remedies, and obligations imposed by this chapter are cumulative to the duties, remedies, or obligations imposed under other law and shall not be construed to relieve an operator from any duties, remedies, or obligations imposed under any other law. SEC. 2. The provisions of this act are severable. If any provision of this act or its application is held invalid, that invalidity shall not affect other provisions or applications that can be given effect without the invalid provision or application.