Recommended guardrail
Do not profile people from AI-inferred emotions or sensitive traits without notice and opt-in
Advisory. Do not store emotions, moods, sentiment, or mental states, or sensitive traits (health (including pregnancy and disability), sexual orientation or gender identity, religion or beliefs, political views or trade union membership, racial or ethnic origin), that an AI infers from someone's messages, voice, face, or behaviour in a profile, CRM, or analytics identity, or use them to target, personalise, rank, price, prioritise, or decide about that person, unless they were told and opted in to that use. Routing or prioritising a person by an inferred emotion or sentiment (a call-back list, queue position, or escalation to a different service tier) counts, even within the same contact. Escalating self-harm or crisis risk, as the crisis-protocol guardrail recommends, is a safety use, not profiling, and so is a response that only adapts the current reply or makes a crisis referral and keeps nothing, and a safety-only escalation record. Detect emotion or sensitive-trait inference (emotion APIs and classifiers, prompts asking a model to infer these traits) whose output reaches a profile, analytics identity, ad audience, or a ranking, pricing, routing, or eligibility input with no consent check.
This is TwinEthos's opinion of what a responsible AI integration does anyway. It is never a legal or standards requirement; where binding law applies, the law governs. Ethical-use guardrails are optional practices, never reported as violations.
The recommended-guardrail rule files are open under CC BY 4.0; attribution and scope are in the terms.
Informational data, not legal advice. Summaries are TwinEthos's own words and rules have not been reviewed by a lawyer: check the official text before relying on any of it. A guard addresses an item; adding it is not a statement that your code meets any law.
Evidence grade
Related law in force in 2 jurisdictions · 3 graded incidents.
Advisory ethical-use recommendation, not law: an optional practice, never reported as a violation. Where binding law applies, the law governs. No binding law on this control itself is in force in the corpus; binding law in provisions cited as convergence, which cover part of the control or a related one, is in force in 2 jurisdictions (EU, US-IL). 3 graded incidents cited.
Related law in force (cited as convergence; not on this control itself)
- AI must not infer emotions in workplace or education settings (European Union (EU); Article 5(1)(f); cited)
- AI must not categorise people by sensitive traits from biometric data (European Union (EU); Article 5(1)(g); cited)
- Deployers of emotion-recognition/biometric-categorisation systems must notify exposed persons (EU AI Act Art. 50(3)) (European Union (EU); Article 50(3); cited)
- Therapists must not use AI to detect clients' emotions or mental states (Illinois HB 1806) (Illinois (US-IL); 225 ILCS 155/20(b); cited)
- AI analysis of a video interview requires notice, explanation, and consent (Illinois) (Illinois (US-IL); 820 ILCS 42/5; cited)
Family “People cannot opt out of automated decision-making, profiling, or personalization”: binding law on related controls is in force in China (CN), Colorado (US-CO), Connecticut (US-CT), Delaware (US-DE), Florida (US-FL), Indiana (US-IN), Kentucky (US-KY), Maryland (US-MD), Minnesota (US-MN), Montana (US-MT), Nebraska (US-NE), New Hampshire (US-NH), New Jersey (US-NJ), Oregon (US-OR), Rhode Island (US-RI), Tennessee (US-TN), Texas (US-TX), Virginia (US-VA); enacted, not yet applying in California (US-CA). Context only: it does not change this guardrail's grade.
Graded incidents
- Meta says it will use people's interactions with Meta AI to personalize content and ads (2025-10; disclosed by the operator) Meta Newsroom (2025-10-01) · evidence grade: primary
- Microsoft retires Azure Face emotion and identity-attribute inference (2022-06; disclosed by the operator) Microsoft Azure Blog (2022-06-21) · evidence grade: primary
- Hungarian regulator fines a bank for AI analysis of callers' emotions without notice or a way to object (2017-05; confirmed) Nemzeti Adatvédelmi és Információszabadság Hatóság (Hungarian data protection authority), decision NAIH-85-3/2022, English version · evidence grade: primary
The guard to add
Keep AI-inferred emotions and sensitive traits out of profiles, targeting, and decisions unless the person opted in after being told.
Consider inventorying every place a model infers emotions, moods, mental states, or sensitive traits (emotion APIs such as Rekognition detect_faces with emotion attributes or Google Vision face_detection likelihoods, text emotion classifiers, or prompts asking a model to read mood, health, sexual orientation, religion, political views, or ethnicity) and following each output. Where it would be written to a user profile, CRM, or analytics identity, or used for ads, personalisation, ranking, pricing, prioritisation, or eligibility, first check a stored opt-in for that specific use, collected with a notice that says what is inferred and how it is used, and otherwise drop the inference. Prefer not inferring sensitive traits at all. A reply that adapts its tone, or a crisis referral, within the same interaction can use the signal without storing it; safety escalation records stay in a store used only for safety.
Example (Python + OpenAI + CRM), before:
label = client.chat.completions.create(model=MODEL, messages=[
{'role': 'system', 'content': "Classify the customer's emotional state and any health conditions they mention. Reply in JSON."},
{'role': 'user', 'content': transcript}]).choices[0].message.content
crm.update_contact(contact_id, {'mood': json.loads(label)['mood'], 'health_flags': json.loads(label)['conditions']})After:
if customer.consents.get('emotion_insights') is True: # opted in after a notice of what is inferred and why
label = client.chat.completions.create(model=MODEL, messages=[
{'role': 'system', 'content': "Classify the caller's mood as calm, neutral, or upset. Reply with one word."},
{'role': 'user', 'content': transcript}]).choices[0].message.content
callback_queue.add(call_id, mood=label, expires_in=timedelta(days=7))
# no health or other sensitive-trait inference; nothing goes to the CRM profileControl: AI-inferred emotions or sensitive traits used to profile people without notice and opt-in. Engineering guidance, not legal advice.
Why
What a model guesses about someone's feelings, health, beliefs, or identity can be wrong and is invisible to the person it describes; stored in a profile and used to rank, target, or price, it shapes how they are treated without their knowing or being able to object. A responsible integration does not keep or act on such inferences unless the person was told and chose it, and prefers not to infer sensitive traits at all. Hungary's data protection authority fined a bank after finding that AI software assessed callers' emotional states to rank calls and choose whom to call back, without telling callers or letting them object; the decision also records that the bank's own technical file showed the emotion was unrecognisable in most cases. Microsoft retired general-purpose emotion and identity-attribute inference from its face API, citing the lack of consensus on what emotions are and the difficulty of generalizing from expression to emotion across use cases, regions, and demographics, and Meta has said it does not use conversations with its AI about sensitive topics such as health, religion, or sexual orientation to show ads.
Class: ethical use · set: ethical use · maturity: reviewed · confidence: medium · id guardrail.ethics-no-profiling-from-inferred-emotions-or-traits