Automated Decisions

The Employer Bought the Tool. Did It Evaluate the Effects?

On January 1, 2026, Illinois made discriminatory AI use an express civil-rights violation. Its employment-law amendments prohibit covered uses that have the effect of discriminating against protected classes and separately require notice that AI is being used.

The amendment to the Illinois Human Rights Act addresses an increasingly ordinary transaction: an employer buys software to help evaluate people. The vendor supplies the product. The employer puts it to work. A score or recommendation influences an employment decision.

The law directs attention to the consequences of that use. But how would an employer know whether its use produces discriminatory effects? And how would an affected person discover what happened?

Illinois has made the obligation explicit. The harder question is whether the information needed to examine compliance will be available.

The Protection Extends Beyond the Hiring Decision

The amendment covers recruitment, hiring, promotion, renewal of employment, selection for training or apprenticeship, discipline, discharge, tenure, and other employment conditions. It also prohibits ZIP codes as proxies for protected classes. Its AI definition includes generative AI and systems producing recommendations or predictions, as well as decisions. The operative language is framed in terms of employees: it prohibits AI that has the effect of subjecting employees to discrimination, and requires notice to an employee, while the same public act defines employee for this Article to include apprentices and applicants for apprenticeship but not applicants generally. Whether the notice duty reaches a rejected job applicant is unsettled on the text.

That reach matters. A tool can influence someone’s opportunities without issuing a final rejection. It might determine which resumes a recruiter sees, which employees are recommended for training, or whose performance warrants closer review.

The Department of Human Rights describes the law as clarifying and emphasizing existing protections against discriminatory effects while establishing notice requirements. Illinois did not first make employment discrimination unlawful when software became involved. It expressly addressed the use of AI within its civil-rights framework.

A Vendor Can Test a Product Without Testing This Use

Consider a hypothetical employer that purchases a tool to rank applicants for a customer-service position. The vendor says the product has been evaluated for bias. The employer sends interview invitations only to applicants above a selected score.

The vendor’s evaluation and the employer’s decision answer different questions.

The vendor may have studied other employers, different jobs, or a different applicant population. Its results may concern an earlier model version. The customer may choose a cutoff the vendor did not evaluate. Recruiters may make exceptions that change who advances.

None of those possibilities proves discrimination. They explain why a product-level assurance does not establish the effects of a particular deployment.

In the hypothetical, the employer would need to understand what the score measures, how the invitation threshold operates, and who is excluded. A prediction that someone resembles previously successful applicants is not necessarily evidence that the person can perform the job.

The relevant use includes what happens around the model. A recruiter can act on a ranking before reviewing any individual application. A manager can consistently override recommendations for one group. Evaluating the software alone would miss those decisions.

The Effects Need Evidence

Suppose the employer discovers that applicants from one protected group receive interview invitations at a lower rate. That result calls for examination. It does not, by itself, establish every element of unlawful discrimination.

The comparison population matters. An analysis of all applicants answers a different question from an analysis of qualified applicants. Looking only at people who reached the final interview can miss exclusions earlier in the process.

The underlying data matter too. Demographic information may be incomplete. Some applicants may decline to provide it. Estimates introduce their own uncertainty. The employer needs to know what the records can establish before treating a numerical result as an answer.

The explanation offered for the screen also needs examination. Does the score predict a relevant job outcome? Was the model trained on actual performance, or on past hiring choices? Does the employer’s chosen threshold serve the purpose offered to justify it? Would another approach achieve that purpose with less adverse impact?

These are practical questions for evaluating the use. They are not a claim that the Illinois amendment prescribes a particular statistical method or that every difference in outcomes violates the law.

Notice Makes the Use Visible

Notice can reveal something an employee would otherwise have no reason to know: software participated in a consequential decision.

That is useful. It can prompt questions about inaccurate information or the role a recommendation played. It also makes it harder for the organization to treat AI use as an invisible detail of its internal process.

But notice alone supplies little evidence about the effect. Knowing that a tool was used does not reveal its inputs, the cutoff applied, the comparison population, or the pattern of human overrides. An individual usually cannot reconstruct those facts from a rejection or promotion decision.

The amendment assigns IDHR responsibility for any rules necessary to implement and enforce the subdivision, including rules addressing notice circumstances, timing, and means. The department published proposed notice rules on May 15, 2026, opening a comment period set to close June 29, then withdrew and postponed them on June 2, 2026, canceling the scheduled public hearing and citing continued collaboration with other state agencies. No revised timeline has issued. Proposed details must be distinguished from enacted requirements; that withdrawal did not repeal the statute.

The Gap Between an Obligation and a Prevention Process

The amendment itself does not establish a recurring independent bias audit, a standardized impact assessment, or a general right to an individual explanation and human reconsideration. Its central approach is an express discrimination prohibition coupled with notice. Illinois has since addressed effects-based liability more generally: the Civil Rights Safeguard Act, signed July 31, 2026, writes disparate-impact standards into the Human Rights Act for employment, financial credit, and public accommodations, effective June 1, 2027.

That is a meaningful step. It is also a limited prevention framework.

If an employer does not examine outcomes, an unlawful effect can remain undetected. If employees cannot obtain the relevant evidence, a prohibition can be difficult to invoke. If no one records the model version, threshold, and decision history, even a later investigation can struggle to reconstruct the event.

A mandatory audit would not automatically resolve these difficulties. An audit of the vendor’s model could still overlook the employer’s choices. An annual review could miss a consequential change introduced the following month. A numerical summary could conceal which stage excluded people.

The useful question is what an evaluation covers and what happens when it identifies a problem. Someone must be able to investigate the cause, change the practice, and determine whether the change worked.

Employers Need Information Their Vendors May Hold

For the hypothetical employer, a useful review would start with the product actually deployed: its intended use, relevant validation evidence, limitations, and version history. It would also examine the employer’s own thresholds and the decisions that followed.

Some of that information belongs to the vendor. Some exists only within the employer’s process. Neither party can supply the complete account alone.

That makes information access a purchasing issue. Before adopting a tool, an employer should determine whether it can obtain the documentation and records needed to investigate an adverse result. A contractual promise of compliance does not supply missing evidence.

This is a practical recommendation, rather than an additional obligation stated in the amendment. The point is to make the employer’s review possible before a complaint requires it.

This post is for general informational purposes only and does not constitute legal advice. For questions about evaluating the effects of an employment AI tool you did not build, consult qualified employment counsel.

Not sure where to start?

Let’s Find the Right Path Forward.

Whether you have a specific question or want to explore how TrustCraft can help, we’re here to talk.