Binding law — in force
Insurance algorithms must not unfairly discriminate via external data
Colorado life insurers using external consumer data and information sources (ECDIS) or algorithms/predictive models that use ECDIS must establish a documented risk-based governance and risk-management framework to detect and remediate unfair discrimination (esp. by race): governing principles, board oversight, senior-management accountability, a cross-functional governance group, lifecycle policies with training, a consumer-complaint/adverse-decision process, a risk rubric, a versioned model inventory, discrimination testing, model-drift monitoring, vendor-selection processes, and annual reviews. Detect an insurance ECDIS/model decision path with no governance-framework or model-inventory artifact.
Who it applies to
- Duty falls on: insurer
- Systems covered: consequential decision
- Sectors: insurance
- All life insurers authorized to do business in Colorado that use ECDIS or algorithms/predictive models using ECDIS. Effective 2023-11-14; governance framework available to the Division on 2024-12-01 and annually. (Auto/health insurance rulemaking ongoing.)
- Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Build decision prompts and feature sets from an allowlist of decision-relevant fields, and redact protected attributes, known proxies, and free text before the model sees them.
At the prompt builder or feature-assembly step on the consequential-decision path, construct model inputs from an explicit allowlist (FEATURE_ALLOWLIST, APPROVED_FEATURES) instead of passing the whole person record or f-string interpolating its fields. Protected attributes (race, sex, religion, age, disability) and proxies (ZIP or postal code, surname, school, census tract) stay out unless a documented justification and a bias test exist, and free text (cover letters, notes, transcripts) goes through redaction (redact_pii, strip_protected_attributes) first. Log the features used and the model output per decision, and run disparity tests on outcomes; human review lowers the risk but does not replace the allowlist.
Where it goes: 1 application source code, 2 data models, 7 prompt construction, 13 tests and evals.
What this provision adds:
- Keep a documented governance framework with board oversight, a versioned model inventory, discrimination testing (especially by race), model-drift monitoring, a consumer complaint and adverse-decision process, and annual reviews.
- Make the governance framework available to the Division from 2024-12-01 and annually.
Example (Python + OpenAI SDK), before:
prompt = f"Applicant {a.last_name}, age {a.age}, zip {a.zip_code}.\nNotes: {a.applicant_notes}\nApprove the loan?"
resp = client.chat.completions.create(model=MODEL, messages=[{'role': 'user', 'content': prompt}])After:
FEATURE_ALLOWLIST = ['income', 'debt_to_income', 'requested_amount', 'payment_history_months']
features = {k: getattr(a, k) for k in FEATURE_ALLOWLIST}
notes = strip_protected_attributes(a.applicant_notes) # drops names, ages, places, etc.
messages = [{'role': 'system', 'content': LENDING_RUBRIC},
{'role': 'user', 'content': json.dumps({'features': features, 'notes': notes})}]
resp = client.chat.completions.create(model=MODEL, messages=messages)
decision_log.record(a.id, features, resp.choices[0].message.content)Control: Protected or proxy attribute reaches AI decision. The same guard addresses 4 items with binding law in 3 jurisdictions. Engineering guidance, not legal advice.
Standards that recommend the same control
- Personal-attribute inputs to AI financial decisions should be justified; no unjustified systematic disadvantage (MAS FEAT) (MAS FEAT Principles · MAS FEAT Principles — Fairness (Justifiability), Principles 1-2)
Related incidents
No guardrail sits on this exact control; these incidents are cited by guardrails on related controls.
- Meta's automated moderation over-enforced Arabic and under-enforced Hebrew content (BSR due diligence) (2021-05; disclosed by the operator). An independent human rights due diligence by BSR, commissioned and published by Meta on September 22, 2022, found that during the May 2021 Israel-Palestine escalation Arabic content saw greater over-enforcement per user than Hebrew content and Hebrew content greater under-enforcement. BSR attributes this in part to Meta having an Arabic hostile-speech classifier but no Hebrew one, and to Arabic classifiers likely being less accurate for Palestinian Arabic. BSR found no intentional bias but 'various instances of unintentional bias' with different impacts on Palestinian and Arabic-speaking users. Meta committed to implement 10 of BSR's 21 recommendations and said it had since launched a Hebrew hostile-speech classifier. Source: Meta (operator response, 2022-09-22) · evidence grade: primary · cited by Check AI ranking, pricing, moderation, and ad targeting for disparities when features can stand in for protected traits
- Toxicity classifiers rate African American English as more offensive (2019; confirmed). University of Washington researchers reported at ACL 2019 that tweets in African American English and tweets by self-identified African Americans were up to two times more likely to be labelled offensive by hate-speech models trained on widely used datasets, and that Jigsaw's public Perspective API showed similar racial bias, rating AAE phrases as more toxic than non-AAE equivalents. Source: Sap et al., 'The Risk of Racial Bias in Hate Speech Detection', ACL 2019 (original researchers) · evidence grade: primary · cited by Check AI ranking, pricing, moderation, and ad targeting for disparities when features can stand in for protected traits
- Ride-hailing fares higher in Chicago neighborhoods with more non-white residents (researcher audit) (2018-11; alleged (not proven)). George Washington University researchers analysing Chicago's public data on more than 100 million ride-hailing trips from November 2018 to September 2019 report that trips in neighborhoods with larger non-white populations, higher poverty, younger residents and more college-educated residents were significantly associated with higher fares. They attribute this to pricing algorithms learning from demand, supply and trip duration. The finding is an observational association from census-tract data; the pricing models themselves were not examined. Source: Pandey & Caliskan, 'Disparate Impact of Artificial Intelligence Bias in Ridehailing Economy's Price Discrimination Algorithms', AIES 2021 (original researchers) · evidence grade: primary · cited by Check AI ranking, pricing, moderation, and ad targeting for disparities when features can stand in for protected traits
- Google ads suggesting arrest records served more often for Black-identifying names (2012; confirmed). Harvard researcher Latanya Sweeney searched 2,184 racially associated full names on google.com and reuters.com (a Google AdSense host) from September 24 to October 23, 2012 and found ads suggestive of an arrest record appeared more often for Black-identifying first names; on reuters.com a Black-identifying name was 25% more likely to get such an ad (statistically significant). Ads appeared regardless of whether the name had an arrest record in the advertiser's database. The paper does not determine whether the advertiser's templates or Google's click-based ad optimization caused the pattern; the advertiser, Instant Checkmate, told the author it gave Google the same ad text for groups of last names. Source: Sweeney, 'Discrimination in Online Ad Delivery' (original researcher, 2013-01-28) · evidence grade: primary · cited by Check AI ranking, pricing, moderation, and ad targeting for disparities when features can stand in for protected traits
Rule id co-sb21-169.insurance-ecdis-unfair-discrimination · review status: primary source derived