On September 30, 2026, California enacted SB 503, assigning responsibilities for biased impacts from clinical decision support to both the companies developing the software and the healthcare organizations using it. The law generally takes effect January 1, 2027.
A developer can supply favorable performance results. A hospital can receive them. Neither event, by itself, establishes how the tool will affect different patients when it enters the hospital’s workflow.
That is the practical question behind the new requirements: what evidence should exist before use, what should be monitored afterward, and who must respond when a risk becomes apparent? Overall accuracy cannot answer all three.
The Law Targets Biased Impacts in Clinical Care
The enacted text of SB 503 defines a biased impact as an adverse impact on an individual based on protected characteristics, including diminished access to healthcare, quality of care, or outcomes. It incorporates the characteristics listed in California Civil Code section 51(b), including race, sex, disability, medical condition, national origin, and primary language.
Its clinical decision support definition covers AI producing predictions, classifications, recommendations, evaluations, or analyses that aid decisions about timing of care, diagnosis, or treatment. Specified administrative functions are excluded, but only to the extent the independent performance of those activities by the system does not require a professional license. The deployers are healthcare facilities, clinics, and physician or group-practice offices using these systems. This definition serves a state healthcare obligation; it does not determine whether FDA regulates the software as a medical device.
Reliability and bias overlap, but they ask different questions. Does a score reliably identify patients needing attention? Does its use leave a protected patient group with worse access, care, or outcomes? A tool can fail everyone. It can also work well on average while missing important cases in one group.
A Favorable Average Can Conceal a Clinical Problem
Consider a hypothetical hospital evaluating a score that alerts clinicians when a patient is deteriorating. In an evaluation, it flags 90 percent of patients who need escalation. Among patients with a particular disability, it flags only 65 percent. These numbers illustrate the issue; they describe no actual product or reported incident.
The overall result does not explain the gap. The evaluation might contain too few relevant patients. A measurement the model expects might be missing more often. The recorded outcome used to judge accuracy might omit cases where deterioration went unrecognized. The clinical presentation might differ.
The consequences also depend on the workflow. Does a low score delay assessment? Can clinicians recognize deterioration through another route? Are alerts followed by timely care for every group? Testing the prediction and examining the care process are connected tasks.
The disparity calls for investigation. It does not, on its own, establish its cause or prove unlawful discrimination. An absence of subgroup evidence presents a different problem: the organization does not yet know whether the proposed use is adequately supported.
Developers Must Supply Evidence That Deployers Can Use
Under Business and Professions Code Chapter 25.3, commencing with section 22758, both roles must make reasonable efforts to identify systems developed for use by deployers that have known or reasonably foreseeable risks of biased impacts resulting from deployment in health programs or activities. The later duties attach only to the systems identified under that provision. Developers have mitigation and disclosure duties; deployers have continuing monitoring and mitigation duties. A healthcare organization that develops or substantially modifies a system can occupy both roles.
Required developer documentation addresses intended use, training-data types and demographic representativeness where available, performance evaluation, limitations, data governance, risks, recommended use and monitoring, and any other documentation reasonably necessary to help deployers understand the outputs and monitor performance for biased impacts. The statute sets that list as a floor rather than a closed list. It is due upon request or initial sale, whichever is earlier, and with applicable material updates. The law includes a qualified internal-use documentation provision. It does not require a public model card for every system.
For a developer, a useful practical starting point is to define the clinical use and the patients for whom it is intended. What decision does the output influence? Which mistakes can delay care or expose patients to unnecessary intervention? Those consequences should inform the evaluation.
The developer can then explain what its evidence establishes, which groups were evaluated, how uncertain the results are, and which settings remain untested. Where demographic information is unavailable, that limitation should be made explicit. Missing information cannot support a claim that group differences have been ruled out.
The next task is mitigation. Depending on the finding, that can involve improving data or the model, changing a decision threshold, narrowing the intended use, or specifying additional clinical safeguards. The response should address the identified mechanism of harm and be evaluated for unintended effects.
The Hospital Needs a Baseline for Its Proposed Use
A practical compliance process begins by comparing the developer’s evidence with the hospital’s patients, available inputs, and response workflow. This is an implementation approach, not an express statutory command to independently revalidate every purchased model.
The comparison should identify where existing evidence is adequate and where additional evaluation is warranted. A model evaluated in one hospital might encounter different measurement practices, disease patterns, or follow-up capacity elsewhere. A vendor’s report cannot establish facts about a local workflow that it never studied.
Additional evaluation can include checking input completeness, reviewing a sample of cases, testing on suitable local records, or conducting a controlled evaluation before outputs influence care. The method should fit the intended use and the seriousness of a mistaken recommendation. Testing that merely repeats the vendor’s result without addressing a material gap adds little.
The baseline should record the evidence reviewed, limitations, deployment conditions, safeguards, and unresolved questions. It gives later monitoring something concrete to compare against. If an important gap remains, restricting use while obtaining evidence is different from treating the gap as proof of safety.
There Is No Single Statutory Bias Score
SB 503 prescribes no universal numerical threshold, testing protocol, independent-audit requirement, or monitoring interval. It allows relevant consensus standards and appropriate industry-accepted impact assessments to inform developer documentation. It does not designate one fairness calculation as a compliance test.
For the hypothetical deterioration model, an evaluation could compare missed cases and false alarms across relevant groups. It could examine whether predicted risks correspond to observed outcomes within each group, a property called calibration. Sample sizes and uncertainty matter: a reassuring percentage based on a handful of patients establishes little.
Equal alert rates are not necessarily the right objective. Patients can have different underlying clinical risks. Research on fairness and calibration also shows that calibration and equal error rates can conflict under differing outcome rates. The organization should explain why its chosen measures capture the relevant clinical harms rather than select whichever measure looks most favorable.
Data deserve the same scrutiny as the model. A diagnosis in a record is not always a complete measure of disease. Spending is not the same as clinical need. Removing race or disability from the inputs does not remove every pathway through which a system can produce unequal effects.
NIST’s work on harmful AI bias emphasizes that the sources extend beyond training data to institutional and human factors. For healthcare organizations, that means examining who receives an assessment, what information is recorded, how staff use recommendations, and what care follows.
Monitoring Needs an Owner and a Response
Once the system is in use, a practical monitoring plan should identify the measures, responsible staff, review intervals, and findings that trigger investigation. Reviews should account for relevant patient groups and clinical consequences, alongside overall performance.
A software update, a new patient population, a change in input collection, or a revised clinical pathway can justify reassessment. Complaints and clinician reports can reveal problems that aggregate performance measures miss.
The response needs to be operational. Investigate the finding, obtain further evidence, adjust safeguards, restrict or suspend the affected use when warranted, and check whether the intervention worked. Record the finding and response. A dashboard that nobody reviews cannot establish that a risk is being addressed.
