Full text
- This is an unofficial copy. The document has been archived and reformatted in plaintext for AGORA. Footnotes, tables, and similar material may be omitted. For the official text, visit the original source.
THE PEOPLE OF THE STATE OF CALIFORNIA DO ENACT AS FOLLOWS:
SECTION 1. Chapter 22.6 (commencing with Section 22601) is added to Division 8 of the Business and Professions Code, to read:
CHAPTER 22.6. Companion Chatbots
22601. As used in this chapter:
(a) “Artificial intelligence” means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
(b) (1) “Companion chatbot” means an artificial intelligence system with a natural language interface that provides adaptive, human-like responses to user inputs and is capable of meeting a user’s social needs, including by exhibiting anthropomorphic features and being able to sustain a relationship across multiple interactions.
(2) “Companion chatbot” does not include any of the following:
(A) A bot that is used only for customer service, a business’ operational purposes, productivity and analysis related to source information, internal research, or technical assistance.
(B) A bot that is a feature of a video game and is limited to replies related to the video game that cannot discuss topics related to mental health, self-harm, sexually explicit conduct, or maintain a dialogue on other topics unrelated to the video game.
(C) A stand-alone consumer electronic device that functions as a speaker and voice command interface, acts as a voice-activated virtual assistant, and does not sustain a relationship across multiple interactions or generate outputs that are likely to elicit emotional responses in the user.
(c) “Companion chatbot platform” means a platform that allows a user to engage with companion chatbots.
(d) “Office” means the Office of Suicide Prevention established pursuant to Section 131300 of the Health and Safety Code.
(e) “Operator” means a person who makes a companion chatbot platform available to a user in the state.
(f) “Sexually explicit conduct” has the meaning defined in Section 2256 of Title 18 of the United States Code.
(g) “Video game” means a game played on an electronic amusement device that utilizes a computer, microprocessor, or similar electronic circuitry and its own monitor, or is designed to be used with a television set or a computer monitor, that interacts with the user of the device.
22602. (a) If a reasonable person interacting with a companion chatbot would be misled to believe that the person is interacting with a human, an operator shall issue a clear and conspicuous notification indicating that the companion chatbot is artificially generated and not human.
(b) (1) An operator shall prevent a companion chatbot on its companion chatbot platform from engaging with users unless the operator maintains a protocol for preventing the production of suicidal ideation, suicide, or self-harm content to the user, including, but not limited to, by providing a notification to the user that refers the user to crisis service providers, including a suicide hotline or crisis text line, if the user expresses suicidal ideation, suicide, or self-harm.
(2) The operator shall publish details on the protocol required by this subdivision on the operator’s internet website.
(c) An operator shall, for a user that the operator knows is a minor, do all of the following:
(1) Disclose to the user that the user is interacting with artificial intelligence.
(2) Provide by default a clear and conspicuous notification to the user at least every three hours for continuing companion chatbot interactions that reminds the user to take a break and that the companion chatbot is artificially generated and not human.
(3) Institute reasonable measures to prevent its companion chatbot from producing visual material of sexually explicit conduct or directly stating that the minor should engage in sexually explicit conduct.
22603. (a) Beginning July 1, 2027, an operator shall annually report to the office all of the following:
(1) The number of times the operator has issued a crisis service provider referral notification pursuant to Section 22602 in the preceding calendar year.
(2) Protocols put in place to detect, remove, and respond to instances of suicidal ideation by users.
(3) Protocols put in place to prohibit a companion chatbot response about suicidal ideation or actions with the user.
(b) The report required by this section shall include only the information listed in subdivision (a) and shall not include any identifiers or personal information about users.
(c) The office shall post data from a report required by this section on its internet website.
(d) An operator shall use evidence-based methods for measuring suicidal ideation.
22604. An operator shall disclose to a user of its companion chatbot platform, on the application, the browser, or any other format that a user can use to access the companion chatbot platform, that companion chatbots may not be suitable for some minors.
22605. A person who suffers injury in fact as a result of a violation of this chapter may bring a civil action to recover all of the following relief:
(a) Injunctive relief.
(b) Damages in an amount equal to the greater of actual damages or one thousand dollars ($1,000) per violation.
(c) Reasonable attorney’s fees and costs.
22606. The duties, remedies, and obligations imposed by this chapter are cumulative to the duties, remedies, or obligations imposed under other law and shall not be construed to relieve an operator from any duties, remedies, or obligations imposed under any other law.
SEC. 2. The provisions of this act are severable. If any provision of this act or its application is held invalid, that invalidity shall not affect other provisions or applications that can be given effect without the invalid provision or application.