TwinEthosRequest access

Control

AI-inferred emotions or sensitive traits used to profile people without notice and opt-in

Emotions, moods, sentiment, or mental states, and sensitive traits (health, including pregnancy and disability; sexual orientation or gender identity; religion or beliefs; political views or trade union membership; racial or ethnic origin) that an AI infers about a person from their messages, voice, face, or behaviour are not stored in a profile, CRM, or analytics identity, and are not used to target, personalise, rank, price, prioritise, or decide about that person, unless the person was told and opted in to that use. Routing or prioritising a person by an inferred emotion or sentiment (a call-back list, queue position, or escalation to a different service tier) is in scope even within the same contact. Escalating self-harm or crisis risk, as the crisis-protocol guardrail recommends, is a safety use and outside this control, as are a response that only adapts the current reply or makes a crisis referral without keeping or reusing the inference, and safety escalation records kept only for safety. Narrower controls cover emotion recognition in workplaces, education, and therapy, biometric categorisation, and the notice to people exposed to emotion recognition, where specific law applies.

Informational data, not legal advice. Summaries are TwinEthos's own words and rules have not been reviewed by a lawyer: check the official text before relying on any of it. A guard addresses an item; adding it is not a statement that your code meets any law.

Family: People cannot opt out of automated decision-making, profiling, or personalization · control id cond.ai-inferred-emotion-or-sensitive-trait-profiling

Reach

1items this one guard addresses
0jurisdictions where binding law on it is in force
0more where it is enacted, not yet applying
0standards and frameworks on the same control

The guard to add

Keep AI-inferred emotions and sensitive traits out of profiles, targeting, and decisions unless the person opted in after being told.

Consider inventorying every place a model infers emotions, moods, mental states, or sensitive traits (emotion APIs such as Rekognition detect_faces with emotion attributes or Google Vision face_detection likelihoods, text emotion classifiers, or prompts asking a model to read mood, health, sexual orientation, religion, political views, or ethnicity) and following each output. Where it would be written to a user profile, CRM, or analytics identity, or used for ads, personalisation, ranking, pricing, prioritisation, or eligibility, first check a stored opt-in for that specific use, collected with a notice that says what is inferred and how it is used, and otherwise drop the inference. Prefer not inferring sensitive traits at all. A reply that adapts its tone, or a crisis referral, within the same interaction can use the signal without storing it; safety escalation records stay in a store used only for safety.

Where it goes: 1 application source code, 2 data models, 7 prompt construction, 9 AI output handling, 10 logs and telemetry.

What reviewers look for: no data flow from an emotion or sensitive-trait inference (rekognition.detect_faces with Attributes ['ALL'] or ['EMOTIONS'], ImageAnnotatorClient.face_detection likelihoods, an emotion text classifier, or a prompt asking a model to infer mood, mental state, health, sexual orientation, religion, political views, or ethnicity) into a profile or CRM update, analytics identify call, ad audience, or a ranking, pricing, routing, or eligibility input, unless a purpose-specific opt-in (for example consents.emotion_insights) is checked first; in-session safety uses that write nothing to the profile.

Example (Python + OpenAI + CRM), before:

label = client.chat.completions.create(model=MODEL, messages=[
    {'role': 'system', 'content': "Classify the customer's emotional state and any health conditions they mention. Reply in JSON."},
    {'role': 'user', 'content': transcript}]).choices[0].message.content
crm.update_contact(contact_id, {'mood': json.loads(label)['mood'], 'health_flags': json.loads(label)['conditions']})

After:

if customer.consents.get('emotion_insights') is True:   # opted in after a notice of what is inferred and why
    label = client.chat.completions.create(model=MODEL, messages=[
        {'role': 'system', 'content': "Classify the caller's mood as calm, neutral, or upset. Reply with one word."},
        {'role': 'user', 'content': transcript}]).choices[0].message.content
    callback_queue.add(call_id, mood=label, expires_in=timedelta(days=7))
# no health or other sensitive-trait inference; nothing goes to the CRM profile

Engineering guidance, not legal advice. Each provision below may add its own details (a cadence, a deadline, a required notice element): open it for those.

Every rule this guard addresses

TwinEthos recommendation (not law) (1)

Related incidents

  • Meta says it will use people's interactions with Meta AI to personalize content and ads (2025-10; disclosed by the operator). On October 1, 2025 Meta announced that from December 16, 2025, in most regions, it would use people's interactions with AI at Meta to personalize the content and ads they see, with notifications to users starting October 7, 2025. Meta says that when people have conversations with Meta AI about topics such as their religious views, sexual orientation, political views, health, racial or ethnic origin, philosophical beliefs, or trade union membership, it does not use those topics to show them ads, and it points people to Ads Preferences and feed controls to adjust what they see. The entry records the operator's own description of its practice. Source: Meta Newsroom (2025-10-01) · evidence grade: primary · cited by Do not profile people from AI-inferred emotions or sensitive traits without notice and opt-in
  • Microsoft retires Azure Face emotion and identity-attribute inference (2022-06; disclosed by the operator). On June 21, 2022 Microsoft said it would retire Azure Face capabilities that infer emotional states and identity attributes such as gender, age, smile, facial hair, hair, and makeup: unavailable to new customers from that day, with existing customers given until June 30, 2023 to stop using them. Microsoft cited privacy, the lack of consensus on a definition of 'emotions', and the inability to generalize the link between facial expression and emotional state across use cases, regions, and demographics, and said that access to capabilities predicting sensitive attributes opens ways to misuse them, including stereotyping, discrimination, or unfair denial of services. It kept these capabilities for controlled accessibility scenarios such as Seeing AI. Source: Microsoft Azure Blog (2022-06-21) · evidence grade: primary · cited by Do not profile people from AI-inferred emotions or sensitive traits without notice and opt-in
  • Hungarian regulator fines a bank for AI analysis of callers' emotions without notice or a way to object (2017-05; confirmed). In decision NAIH-85-3/2022 of 8 February 2022, Hungary's data protection authority found that Budapest Bank's speech-analysis software, which the bank said it introduced on 26 May 2017, automatically analysed recorded customer-service calls for keywords and for the emotional state of the caller and the employee, and that the results were used to rank calls and to select dissatisfied customers to call back. The Authority found that callers were not told at the start of calls about the voice analysis, the automatic evaluation of their emotions, or the resulting possible callback, and could not object; it rejected the bank's statement that the software contained no artificial intelligence. It found infringements of GDPR Articles 5(1)(a)-(b), 6(1), 6(4), 12(1), 13, 21(1)-(2), 24(1) and 25(1), ordered the bank not to analyse emotions in the voice analysis, and imposed a fine of HUF 250 million. The decision also records, from the bank's own technical file, that the emotion was unrecognisable in 91.96% of cases. Source: Nemzeti Adatvédelmi és Információszabadság Hatóság (Hungarian data protection authority), decision NAIH-85-3/2022, English version · evidence grade: primary · cited by Do not profile people from AI-inferred emotions or sensitive traits without notice and opt-in