Full text
- This is an unofficial copy. The document has been archived and reformatted in plaintext for AGORA. Footnotes, tables, and similar material may be omitted. For the official text, visit the original source.
- Thematic tags for this document are awaiting validation (peer review by a second AGORA editor).
LONG TITLE
General Description:
This bill enacts provisions relating to the regulation of mental health chatbots that use
artificial intelligence technology.
Highlighted Provisions:
This bill:
▸ defines terms;
▸ establishes protections for users of mental health chatbots that use artificial intelligence
technology;
▸ prohibits certain uses of personal information by a mental health chatbot;
▸ requires a mental health chatbot to make certain disclosures to users;
▸ provides enforcement authority to the Division of Consumer Protection;
▸ establishes requirements for creating and maintaining policies for mental health chatbots;
▸ creates rebuttable presumptions for suppliers who comply with policy requirements; and
▸ provides a severability clause.
Money Appropriated in this Bill:
None
Other Special Clauses:
None
Utah Code Sections Affected:
AMENDS:
13-2-1, as last amended by Laws of Utah 2024, Chapter 132
ENACTS:
13-72a-101, Utah Code Annotated 1953
13-72a-201, Utah Code Annotated 1953
13-72a-202, Utah Code Annotated 1953
13-72a-203, Utah Code Annotated 1953
13-72a-204, Utah Code Annotated 1953
13-72a-301, Utah Code Annotated 1953
58-60-118, Utah Code Annotated 1953
Be it enacted by the Legislature of the state of Utah:
Section 1. Section 13-2-1 is amended to read:
13-2-1 . Consumer protection division established -- Functions.
(1) There is established within the Department of Commerce the Division of Consumer Protection
(2) The division shall administer and enforce the following:
(a) Chapter 10a, Music Licensing Practices Act;
(b) Chapter 11, Utah Consumer Sales Practices Act;
(c) Chapter 15, Business Opportunity Disclosure Act;
(d) Chapter 20, New Motor Vehicle Warranties Act;
(e) Chapter 21, Credit Services Organizations Act;
(f) Chapter 22, Charitable Solicitations Act;
(g) Chapter 23, Health Spa Services Protection Act;
(h) Chapter 25a, Telephone and Facsimile Solicitation Act;
(i) Chapter 26, Telephone Fraud Prevention Act;
(j) Chapter 28, Prize Notices Regulation Act;
(k) Chapter 32a, Pawnshop, Secondhand Merchandise, and Catalytic Converter
Transaction Information Act;
(l) Chapter 34, Utah Postsecondary School and State Authorization Act;
(m) Chapter 41, Price Controls During Emergencies Act;
(n) Chapter 42, Uniform Debt-Management Services Act;
(o) Chapter 49, Immigration Consultants Registration Act;
(p) Chapter 51, Transportation Network Company Registration Act;
(q) Chapter 52, Residential Solar Energy Disclosure Act;
(r) Chapter 53, Residential, Vocational and Life Skills Program Act;
(s) Chapter 54, Ticket Website Sales Act;
(t) Chapter 56, Ticket Transferability Act;
(u) Chapter 57, Maintenance Funding Practices Act;
(v) Chapter 61, Utah Consumer Privacy Act;
[(w) Chapter 63, Utah Social Media Regulation Act;]
[(x)] (w) Chapter 64, Vehicle Value Protection Agreement Act;
[(y)] (x) Chapter 65, Utah Commercial Email Act;
[(z)] (y) Chapter 67, Online Dating Safety Act;
[(aa)] (z) Chapter 68, Lawyer Referral Consultants Registration Act;
[(bb)] (aa) Chapter 70, Automatic Renewal Contracts Act[; and] ;
[(cc)] (bb) Chapter 71, Utah Minor Protection in Social Media Act[.] ; and
(cc) Chapter 72a, Artificial Intelligence Applications Relating to Mental Health.
(3) In accordance with Title 63G, Chapter 3, Utah Administrative Rulemaking Act,
the division may make rules to establish:
(a) a public list that identifies a person who:
(i) violates a chapter described in Subsection (2);
(ii) without proper legal justification, fails to comply with an order, subpoena,
judgment, or other legal process issued by:
(A) the division; or
(B) a court of competent jurisdiction; or
(iii) breaches a settlement agreement, stipulation, assurance of voluntary compliance,
or similar instrument signed by the person and the division; and
(b) a process by which a person may be removed from the list the division establishes as
described in Subsection (3)(a).
Section 2. Section 13-72a-101 is enacted to read:
CHAPTER 72a. ARTIFICIAL INTELLIGENCE APPLICATIONS RELATING TO
MENTAL HEALTH.
Part 1. General Provisions
13-72a-101 . Definitions.
As used in this chapter:
(1) "Artificial intelligence" means the same as that term is defined in Section 13-72-101.
(2) "Artificial intelligence technology" means the same as that term is defined in Section
13-72-101.
(3) "Confidential communications" means the same as that term is defined in Section
58-60-102.
(4) "Covered entity" means the same as that term is defined in 45 C.F.R. Sec. 160.103.
(5)Division" means the Division of Consumer Protection created in Section 13-2-1.
(6) "Generative artificial intelligence" means an artificial system that:
(a) is trained on data;
(b) interacts with a person using text, audio, or visual communication; and
(c) generates non-scripted outputs similar to outputs created by a human, with limited or
no human oversight.
(7) Health care provider" means the same as that term is defined in 45 C.F.R. Sec. 160.103.
(8) "Health plan" means the same as that term is defined in 45 C.F.R. Sec. 160.103.
(9) "Individually identifiable health information" means any information, whether oral or
recorded in any form or medium, that relates to the physical or mental health or
condition of an individual.
(10)(a) "Mental health chatbot" means an artificial intelligence technology that:
(i) uses generative artificial intelligence to engage in interactive conversations with a
user of the mental health chatbot similar to the confidential communications that
an individual would have with a licensed mental health therapist; and
(ii) a supplier represents, or a reasonable person would believe, can or will provide
mental health therapy or help a user manage or treat mental health conditions.
(b) "Mental health chatbot" does not include artificial intelligence technology that only:
(i) provides scripted output, such as guided meditations or mindfulness exercises; or
(ii) analyzes an individual's input for the purpose of connecting the individual with a
human mental health therapist.
(11) "Mental health therapist" means the same as that term is defined in Section 58-60-102.
(12) "Personal data" means the same as term is defined Section 63A-19-101.
(13) "Scientific research development" means research:
(a) conducted by a researcher affiliated with: (i) an institution of higher education;
(ii) a research organization; or
(iii) a healthcare facility; and
(b) that is:
(i) approved by an institutional review board; and
(ii) conducted in accordance with applicable ethics requirements for human subject
research.
(14) "Supplier" means the same as that term is defined in Section 13-11-3.
(15) "Utah user" means an individual located in the state at the time the individual accesses or
uses a mental health chatbot.
(16) "User input" means content provided to a mental health chatbot by a Utah user.
Section 3. Section 13-72a-201 is enacted to read:
Part 2. Protections for Users of Mental Health Chatbots
13-72a-201 . Protection of personal information.
(1) A supplier of a mental health chatbot may not sell to or share with any third party any:
(a) individually identifiable health information of a Utah user; or
(b) user input of a Utah user.
(2) Subsection (1) does not apply to:
(a) individually identifiable health information:
(i) requested by a health care provider with consent of the Utah user; or
(ii) provided to a health plan of a Utah user upon request of the Utah user; or
(b) personal data provided by a supplier to an associated third party in the ordinary
course of business, provided that the supplier complies with the security and privacy
provisions of 45 C.F.R. Part 160 and 45 C.F.R. Part 164, Subparts A, C, and E, to the
same extent as is required of a covered entity.
@ Section 4. Section 13-72a-202 is enacted to read:
13-72a-202 . Restrictions on advertising.
(1) A supplier may not use a mental health chatbot to advertise a specific product or service
to a Utah user in a conversation between the Utah user and the mental health chatbot
unless the mental health chatbot:
(a) clearly and conspicuously identifies the advertisement as an advertisement; and
(b) clearly and conspicuously discloses to the Utah user any:
(i) sponsorship;
(ii) business affiliation; or
(iii) agreement that the supplier has with a third party to promote, advertise, or
recommend the product or service.
(2) A supplier of a mental health chatbot may not use a Utah user's input to:
(a) determine whether to display an advertisement for a product or service to the Utah
user, unless the advertisement is for the mental health chatbot itself;
(b) determine a product, service, or category of product or service, to advertise to the
Utah user; or
(c) customize how an advertisement is presented to the Utah user.
(3) This section does not prohibit a mental health chatbot from recommending that a Utah
user seek counseling, therapy, or other assistance from a licensed professional, including
a specific licensed professional.
Section 5. Section 13-72a-203 is enacted to read:
13-72a-203 . Disclosure requirements.
(1) A supplier of a mental health chatbot shall cause the mental health chatbot to clearly
and conspicuously disclose to a Utah user that the mental health chatbot is an artificial
intelligence technology and not a human.
(2) The disclosure described in Subsection (1) shall be made:
(a) before the Utah user may access the features of the mental health chatbot;
(b) at the beginning of any interaction with the Utah user if the Utah user has not
accessed the mental health chatbot within the previous seven days; and
(c) any time a Utah user asks or otherwise prompts the mental health chatbot about
whether artificial intelligence is being used.
Section 6. Section 13-72a-204 is enacted to read:
13-72a-204 . Violations -- Enforcement authority.
(1) The division shall administer and enforce the provisions of this chapter in accordance
with Chapter 2, Division of Consumer Protection.
(2) The attorney general shall:
(a) upon request, give legal advice to the division; and
(b) act as counsel for the division in the exercise of the division's responsibilities under
this chapter.
(3) In addition to the division's enforcement powers under Chapter 2, Division of Consumer
Protection:
(a) the division director may impose an administrative fine of up to $2,500 for each
violation of this chapter; and
(b) the division may bring an action in a court of competent jurisdiction to enforce a
provision of this chapter.
(4) In a court action by the division to enforce a provision of this chapter, the court may:
(a) declare that an act or practice violates a provision of this chapter;
(b) issue an injunction for a violation of this part;
(c) order disgorgement of money received in violation of this chapter;
(d) order payment of disgorged money to an injured purchaser or consumer;
(e) impose a fine of up to $2,500 for each violation of this chapter; or
(f) award other relief that the court determines reasonable and necessary.
(5) If a court awards judgment or injunctive relief to the division, the court shall award the division:
(a) reasonable attorney fees;
(b) court costs; and
(c) investigative fees.
(6) A court may impose a civil penalty of no more than $5,000 for each violation of an
administrative or court order issued for a violation of this chapter.
(7) The attorney general may bring a civil action on behalf of the division to collect a civil
penalty imposed under this section.
(8) The division shall deposit all fines and civil penalties collected under this section into
the Consumer Protection Education and Training Fund created in Section 13-2-8.
Section 7. Section 13-72a-301 is enacted to read:
Part 3. Severability
13-72a-301 . Severability
(1) If any provision of this chapter or the application of any provision of this chapter to any
person or circumstance is held invalid by a final decision of a court of competent
jurisdiction, the remainder of this chapter shall be given effect without the invalid
provision or application.
(2) The provisions of this chapter are severable.
Section 8. Section 58-60-118 is enacted to read:
58-60-118 . Mental health chatbots -- Affirmative defense.
(1) As used in this section:
(a) "Mental health chatbot" means the same as that term is defined in Section 13-72a-101.
(b) "Supplier" means the same as that term is defined in Section 13-11-3.
(2) It is an affirmative defense to liability in an action brought under Subsection 58-1-501(1)
or Subsection 58-1-501(2) if the supplier demonstrates that the supplier:
(a) created, maintained, and implemented a policy that meets the requirements of
Subsection (3);
(b) maintains documentation regarding the development and implementation of the mental health chatbot that describes:
(i) foundation models used in development;
(ii) training data used;
(iii) compliance with federal health privacy regulations;
(iv) user data collection and sharing practices; and
(v) ongoing efforts to ensure accuracy, reliability, fairness, and safety;
(c) filed the policy with the division as described in Subsection (4); and
(d) complied with all requirements of the filed policy at the time of the alleged violation.
(3) A policy described in Subsection (2)(a) must:
(a) be in writing;
(b) clearly state:
(i) the intended purposes of the mental health chatbot; and
(ii) the abilities and limitations of the mental health chatbot; and
(c) describe the procedures by which the supplier:
(i) ensures that licensed mental health therapists are involved in the development and review process;
(ii) ensures the mental health chatbot is developed and monitored in a manner
consistent with clinical best practices;
(iii) conducts testing, prior to making the mental health chatbot publicly available and
regularly thereafter, to ensure that the output of the mental health chatbot poses no
greater risk to a user than that posed to an individual in therapy with a licensed
mental health therapist;
(iv) identifies reasonably foreseeable adverse outcomes to, and potentially harmful
interactions with, users that could result from using the mental health chatbot;
(v) provides a mechanism for a user to report any potentially harmful interactions
from use of the mental health chatbot;
(vi) implements protocols to assess and respond to risk of harm to users or other
individuals;
(vii) details actions taken to prevent or mitigate any such adverse outcomes or
potentially harmful interactions;
(viii) implements protocols to respond in real time to acute risk of physical harm;
(ix) reasonably ensures regular, objective reviews of safety, accuracy, and efficacy,
which may include internal or external audits;
(x) provides users any necessary instructions on the safe use of the mental health
chatbot;
(xi) ensures users understand they are interacting with artificial intelligence;
(xii) ensures users understand the intended purpose, capabilities, and limitations of
the mental health chatbot;
(xiii) prioritizes user mental health and safety over engagement metrics or profit;
(xiv) implements measures to prevent discriminatory treatment of users; and
(xv) ensures compliance with the security and privacy provisions of 45 C.F.R. Part
160 and 45 C.F.R. Part 164, Subparts A, C, and E, as if the supplier were a
covered entity, and applicable consumer protection requirements, including
Sections 13-72a-201, 13-72a-202, and 13-72a-203.
(4) To file a policy with the division under this section, a supplier of a mental health chatbot:
(a) shall provide to the division:
(i) the name and address of the supplier;
(ii) the name of the mental health chatbot supplied by the supplier;
(iii) the written policy described in Subsection (3); and
(iv) a fee set in accordance with Section 63J-1-504;
(b) shall file in a manner established by the division; and
(c) may provide to the division:
(i) any revisions to a policy filed under this section; or
(ii) any other documentation the supplier elects to provide.
(5) The division:
(a) shall provide a means for a supplier of a mental health chatbot to file under this
section; and
(b) may impose an annual filing fee set in accordance with Section 63J-1-504.
(6) The affirmative defense described in this section applies only in an administrative or civil action alleging a violation of:
(a) Subsection 58-1-501(1); or
(b) Subsection 58-1-501(2).
(7) Nothing in this section shall be construed to:
(a) bar the division from bringing an action under Subsection 58-1-501(1) or Subsection
58-1-501(2) against the supplier of a mental health chatbot; or
(b) recognize a mental health chatbot as a licensed mental health therapist.
Section 9. Effective Date.
This bill takes effect on May 7, 2025.