HHS Section 1557 rule, 45 CFR 92.210 (patient care decision support tools)

Binding law — in force

Identify patient care decision support tools that use race, color, national origin, sex, age or disability inputs and mitigate the risk (45 CFR 92.210)

45 CFR 92.210(b) · official text · In force: applies since 1 May 2025 · United States (federal) (US)

A covered entity must not discriminate on the basis of race, color, national origin, sex, age or disability through the use of patient care decision support tools (45 CFR 92.210(a)); it has an ongoing duty to make reasonable efforts to identify uses of such tools that employ input variables or factors measuring those characteristics (92.210(b)) and, for each tool identified, to make reasonable efforts to mitigate the risk of discrimination (92.210(c)). A 'patient care decision support tool' is any automated or non-automated tool used to support clinical decision-making (92.4); the Department states it includes automated decision systems and AI. Detect clinical model code that feeds protected characteristics to a model with no identification or mitigation record.

Who it applies to

• Duty falls on: organization
• Sectors: healthcare, insurance
• Covered entities under Section 1557 (45 CFR 92.4: recipients of HHS Federal financial assistance, HHS, and title I ACA entities, such as hospitals, clinics, health insurance issuers and state Medicaid programs) that use automated or non-automated patient care decision support tools, including AI and predictive models, to support clinical decision-making in their health programs or activities, for patients in the United States. 92.210(a) applies from 2024-07-05; the identification and mitigation duties of 92.210(b)-(c) from 2025-05-01 (300 days after 2024-07-05).
• Not covered:
  • Employers and other plan sponsors of group health plans, with regard to their employment practices, including the provision of employee health benefits (45 CFR 92.2(b))
• Whether it applies depends on facts outside the code; a person has to decide.

prompt = f"Applicant {a.last_name}, age {a.age}, zip {a.zip_code}.\nNotes: {a.applicant_notes}\nApprove the loan?"
resp = client.chat.completions.create(model=MODEL, messages=[{'role': 'user', 'content': prompt}])

FEATURE_ALLOWLIST = ['income', 'debt_to_income', 'requested_amount', 'payment_history_months']

features = {k: getattr(a, k) for k in FEATURE_ALLOWLIST}
notes = strip_protected_attributes(a.applicant_notes)   # drops names, ages, places, etc.
messages = [{'role': 'system', 'content': LENDING_RUBRIC},
            {'role': 'user', 'content': json.dumps({'features': features, 'notes': notes})}]
resp = client.chat.completions.create(model=MODEL, messages=messages)
decision_log.record(a.id, features, resp.choices[0].message.content)

Standards that recommend the same control

• Personal-attribute inputs to AI financial decisions should be justified; no unjustified systematic disadvantage (MAS FEAT) (MAS FEAT Principles · MAS FEAT Principles — Fairness (Justifiability), Principles 1-2)

Related incidents

No guardrail sits on this exact control; these incidents are cited by guardrails on related controls.

• Meta's automated moderation over-enforced Arabic and under-enforced Hebrew content (BSR due diligence) (2021-05; disclosed by the operator). An independent human rights due diligence by BSR, commissioned and published by Meta on September 22, 2022, found that during the May 2021 Israel-Palestine escalation Arabic content saw greater over-enforcement per user than Hebrew content and Hebrew content greater under-enforcement. BSR attributes this in part to Meta having an Arabic hostile-speech classifier but no Hebrew one, and to Arabic classifiers likely being less accurate for Palestinian Arabic. BSR found no intentional bias but 'various instances of unintentional bias' with different impacts on Palestinian and Arabic-speaking users. Meta committed to implement 10 of BSR's 21 recommendations and said it had since launched a Hebrew hostile-speech classifier. Source: Meta (operator response, 2022-09-22) · evidence grade: primary · cited by Check AI ranking, pricing, moderation, and ad targeting for disparities when features can stand in for protected traits, and screen every served language
• Toxicity classifiers rate African American English as more offensive (2019; confirmed). University of Washington researchers reported at ACL 2019 that tweets in African American English and tweets by self-identified African Americans were up to two times more likely to be labelled offensive by hate-speech models trained on widely used datasets, and that Jigsaw's public Perspective API showed similar racial bias, rating AAE phrases as more toxic than non-AAE equivalents. Source: Sap et al., 'The Risk of Racial Bias in Hate Speech Detection', ACL 2019 (original researchers) · evidence grade: primary · cited by Check AI ranking, pricing, moderation, and ad targeting for disparities when features can stand in for protected traits, and screen every served language
• Ride-hailing fares higher in Chicago neighborhoods with more non-white residents (researcher audit) (2018-11; alleged (not proven)). George Washington University researchers analysing Chicago's public data on more than 100 million ride-hailing trips from November 2018 to September 2019 report that trips in neighborhoods with larger non-white populations, higher poverty, younger residents and more college-educated residents were significantly associated with higher fares. They attribute this to pricing algorithms learning from demand, supply and trip duration. The finding is an observational association from census-tract data; the pricing models themselves were not examined. Source: Pandey & Caliskan, 'Disparate Impact of Artificial Intelligence Bias in Ridehailing Economy's Price Discrimination Algorithms', AIES 2021 (original researchers) · evidence grade: primary · cited by Check AI ranking, pricing, moderation, and ad targeting for disparities when features can stand in for protected traits, and screen every served language
• Google ads suggesting arrest records served more often for Black-identifying names (2012; confirmed). Harvard researcher Latanya Sweeney searched 2,184 racially associated full names on google.com and reuters.com (a Google AdSense host) from September 24 to October 23, 2012 and found ads suggestive of an arrest record appeared more often for Black-identifying first names; on reuters.com a Black-identifying name was 25% more likely to get such an ad (statistically significant). Ads appeared regardless of whether the name had an arrest record in the advertiser's database. The paper does not determine whether the advertiser's templates or Google's click-based ad optimization caused the pattern; the advertiser, Instant Checkmate, told the author it gave Google the same ad text for groups of last names. Source: Sweeney, 'Discrimination in Online Ad Delivery' (original researcher, 2013-01-28) · evidence grade: primary · cited by Check AI ranking, pricing, moderation, and ad targeting for disparities when features can stand in for protected traits, and screen every served language

Rule id us-hhs-1557-decision-support-tools.identify-and-mitigate-protected-attribute-inputs · review status: primary source derived