Illinois SB2203 (Preventing Algorithmic Discrimination Act)

A BILL FOR AN ACT concerning business. Be it enacted by the People of the State of Illinois, represented in the General Assembly: Section 1. Short title. This Act may be cited as the Preventing Algorithmic Discrimination Act. Section 5. Definitions. As used in this Act: "Algorithmic discrimination" means the condition in which an automated decision tool contributes to unjustified differential treatment or impacts disfavoring people based on their actual or perceived race, color, ethnicity, sex, religion, age, national origin, limited English proficiency, disability, veteran status, genetic information, reproductive health, or any other classification protected by State law. "Algorithmic discrimination" does not include: (1) the offer, license, or use of a high-risk artificial intelligence system by a deployer for the sole purpose of: (A) the deployer's self-testing to identify, mitigate, or prevent discrimination or otherwise ensure compliance with state and federal law; or (B) expanding an applicant, customer, or participant pool to increase diversity or redress historical discrimination; or (2) an act or omission by or on behalf of a private club or other establishment that is not in fact open to the public, as set forth in the Civil Rights Act of 1964. "Artificial intelligence system" means a machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments. "Artificial intelligence system" includes a generative artificial intelligence system. For the purposes of this definition, "generative artificial intelligence system" means an automated computing system that, when prompted with human prompts, descriptions, or queries, can produce outputs that simulate human-produced content, including, but not limited to: (1) textual outputs, such as short answers, essays, poetry, or longer compositions or answers; (2) image outputs, such as fine art, photographs, conceptual art, diagrams, and other images; (3) multimedia outputs, such as audio or video in the form of compositions, songs, or short-form or long-form audio or video; and (4) other content that would otherwise be produced by human means "Automated decision tool" means a system or service that uses artificial intelligence and has been specifically developed and marketed to, or specifically modified to, make, or be a controlling factor in making, consequential decisions. "Consequential decision" means a decision or judgment that has a legal, material, or similarly significant effect on an individual's life relating to the impact of, access to, or the cost, terms, or availability of, any of the following: (1) employment, worker management, or self-employment, including, but not limited to, all of the following: (A) pay or promotion; (B) hiring or termination; and (C) automated task allocation; (2) education and vocational training, including, but not limited to, all of the following: (A) assessment, including, but not limited to, detecting student cheating or plagiarism; (B) accreditation; (C) certification; (D) admissions; and (E) financial aid or scholarships; (3) housing or lodging, including rental or short-term housing or lodging; (4) essential utilities, including electricity, heat, water, Internet or telecommunications access, or transportation; (5) family planning, including adoption services or reproductive services, as well as assessments related to child protective services; (6) healthcare or health insurance, including mental health care, dental, or vision; (7) financial services, including a financial service provided by a mortgage company, mortgage broker, or creditor; (8) the criminal justice system, including, but not limited to, all of the following: (A) risk assessments for pretrial hearings; (B) sentencing; and (C) parole; (9) legal services, including private arbitration or mediation; (10) voting; and (11) access to benefits or services or assignment of penalties. "Deployer" means a person, partnership, State or local government agency, or corporation that uses an automated decision tool to make a consequential decision. "Impact assessment" means a documented risk-based evaluation of an automated decision tool that meets the criteria of Section 10. "Sex" includes pregnancy, childbirth, and related conditions, gender identity, intersex status, and sexual orientation. "Significant update" means a new version, new release, or other update to an automated decision tool that includes changes to its use case, key functionality, or expected outcomes.

Section 10. Impact assessment. (a) On or before January 1, 2027, and annually thereafter, a deployer of an automated decision tool shall perform an impact assessment for any automated decision tool the deployer uses that includes all of the following: (1) a statement of the purpose of the automated decision tool and its intended benefits, uses, and deployment contexts; (2) a description of the automated decision tool's outputs and how they are used to make, or be a controlling factor in making, a consequential decision; (3) a summary of the type of data collected from natural persons and processed by the automated decision tool when it is used to make, or be a controlling factor in making, a consequential decision; (4) an analysis of potential adverse impacts on the basis of sex, race, color, ethnicity, religion, age, national origin, limited English proficiency, disability, veteran status, or genetic information from the deployer's use of the automated decision tool; (5) a description of the safeguards implemented, or that will be implemented, by the deployer to address any reasonably foreseeable risks of algorithmic discrimination arising from the use of the automated decision tool known to the deployer at the time of the impact assessment; (6) a description of how the automated decision tool will be used by a natural person, or monitored when it is used, to make, or be a controlling factor in making, a consequential decision; and (7) a description of how the automated decision tool has been or will be evaluated for validity or relevance. (b) A deployer shall, in addition to the impact assessment required by subsection (a), perform, as soon as feasible, an impact assessment with respect to any significant update. (c) This Section does not apply to a deployer with fewer than 25 employees unless, as of the end of the prior calendar year, the deployer deployed an automated decision tool that impacted more than 999 people per year.

Section 15. Notification and accommodations. (a) A deployer shall, at or before the time an automated decision tool is used to make a consequential decision, notify any natural person who is the subject of the consequential decision that an automated decision tool is being used to make, or be a controlling factor in making, the consequential decision. A deployer shall provide to a natural person notified under this subsection all of the following: (1) a statement of the purpose of the automated decision tool; (2) the contact information for the deployer; and (3) a plain language description of the automated decision tool that includes a description of any human components and how any automated component is used to inform a consequential decision.

(b) If a consequential decision is made solely based on the output of an automated decision tool, a deployer shall, if technically feasible, accommodate a natural person's request to not be subject to the automated decision tool and to be subject to an alternative selection process or accommodation. After a request is made under this subsection, a deployer may reasonably request, collect, and process information from a natural person for the purposes of identifying the person and the associated consequential decision. If the person does not provide that information, the deployer shall not be obligated to provide an alternative selection process or accommodation.

Section 20. Governance program. (a) A deployer shall establish, document, implement, and maintain a governance program that contains reasonable administrative and technical safeguards to map, measure, manage, and govern the reasonably foreseeable risks of algorithmic discrimination associated with the use or intended use of an automated decision tool. The safeguards required by this subsection shall be appropriate to all of the following: (1) the use or intended use of the automated decision tool; (2) the deployer's role as a deployer; (3) the size, complexity, and resources of the deployer; (4) the nature, context, and scope of the activities of the deployer in connection with the automated decision tool; and (5) the technical feasibility and cost of available tools, assessments, and other means used by a deployer to map, measure, manage, and govern the risks associated with an automated decision tool. (b) The governance program required by this Section shall be designed to do all of the following: (1) identify and implement safeguards to address reasonably foreseeable risks of algorithmic discrimination resulting from the use or intended use of an automated decision tool; (2) if established by a deployer, provide for the performance of impact assessments as required by Section 10; (3) conduct an annual and comprehensive review of policies, practices, and procedures to ensure compliance with this Act; (4) maintain for 2 years after completion the results of an impact assessment; and (5) evaluate and make reasonable adjustments to administrative and technical safeguards in light of material changes in technology, the risks associated with the automated decision tool, the state of technical standards, and changes in business arrangements or operations of the deployer.

(c) A deployer shall designate at least one employee to be responsible for overseeing and maintaining the governance program and compliance with this Act. An employee designated under this subsection shall have the authority to assert to the employee's employer a good faith belief that the design, production, or use of an automated decision tool fails to comply with the requirements of this Act. An employer of an employee designated under this subsection shall conduct a prompt and complete assessment of any compliance issue raised by that employee. (d) This Section does not apply to a deployer with fewer than 25 employees unless, as of the end of the prior calendar year, the deployer deployed an automated decision tool that impacted more than 999 people per year.

Section 25. Public statement of policy. A deployer shall make publicly available, in a readily accessible manner, a clear policy that provides a summary of both of the following: (1) the types of automated decision tools currently in use or made available to others by the deployer; and (2) how the deployer manages the reasonably foreseeable risks of algorithmic discrimination that may arise from the use of the automated decision tools it currently uses or makes available to others.

Section 30. Algorithmic discrimination. (a) A deployer shall not use an automated decision tool that results in algorithmic discrimination. (b) On and after January 1, 2028, a person may bring a civil action against a deployer for violation of this Section. In an action brought under this subsection, the plaintiff shall have the burden of proof to demonstrate that the deployer's use of the automated decision tool resulted in algorithmic discrimination that caused actual harm to the person bringing the civil action. (c) In addition to any other remedy at law, a deployer that violates this Section shall be liable to a prevailing plaintiff for any of the following: (1) compensatory damages; (2) declaratory relief; and (3) reasonable attorney's fees and costs.

Section 35. Impact assessment. (a) Within 60 days after completing an impact assessment required by this Act, a deployer shall provide the impact assessment to the Attorney General. (b) A deployer who knowingly violates this Section shall be liable for an administrative fine of not more than $10,000 per violation in an administrative enforcement action brought by the Attorney General. Each day on which an automated decision tool is used for which an impact assessment has not been submitted as required under this Section shall give rise to a distinct violation of this Section. (c) The Attorney General may share impact assessments with other State entities as appropriate.

Section 40. Enforcement. A violation of this Act constitutes an unlawful practice under the Consumer Fraud and Deceptive Business Practices Act. All remedies, penalties, and authority granted to the Attorney General by the Consumer Fraud and Deceptive Business Practices Act shall be available to him or her for the enforcement of this Act.

Section 95. The Consumer Fraud and Deceptive Business Practices Act is amended by adding Section 2HHHH as follows: (815 ILCS 505/2HHHH new) Sec. 2HHHH. Violations of the Preventing Algorithmic Discrimination Act. A person who violates the Preventing Algorithmic Discrimination Act commits an unlawful practice within the meaning of this Act.