European Commission / national market surveillance authorities · European Union (EU) · 16 provisions encoded · verified against the official source as of 2026-09-26.
Informational data, not legal advice. Summaries are TwinEthos's own words and rules have not been reviewed by a lawyer: check the official text before relying on any of it. A guard addresses an item; adding it is not a statement that your code meets any law.
AI must not categorise people by sensitive traits from biometric data
Article 5(1)(g) · official text · In force: applies since 2 Feb 2025 · European Union (EU)
Using biometric data to deduce or infer sensitive traits (race, political opinions, trade-union membership, religious/philosophical beliefs, sex life, sexual orientation) is prohibited. Detect inference of these categories from biometric input.
Who it applies to
Duty falls on: provider, deployer
Providers/deployers of biometric categorisation inferring sensitive traits, affecting persons in the EU.
The guard to add
Remove any model, prompt, or label set that infers race, political opinion, union membership, religion or beliefs, sex life, or sexual orientation from biometric data.
In biometric pipelines (face, voice, gait, iris), the code computes only the attributes the feature needs and never a sensitive-trait category: no classifier heads or labels for race or ethnicity, political opinion, trade-union membership, religious or philosophical belief, sex life, or sexual orientation, and no attribute API used for those traits (for example DeepFace.analyze with the 'race' action). Prompts that send a person's photo or voice to a multimodal model instruct it not to guess these traits, and an output check strips any such inference. The data model has no columns for these traits derived from biometric input.
Where it goes: 1 application source code, 2 data models, 7 prompt construction, 9 AI output handling.
Example (Python DeepFace), before:
result = DeepFace.analyze(img_path=photo, actions=['age', 'gender', 'race'])
profile.update(age=result[0]['age'], ethnicity=result[0]['dominant_race'])
After:
result = DeepFace.analyze(img_path=photo, actions=['age']) # no 'race' action
profile.update(age=result[0]['age'])
Rule id eu-ai-act.art5.biometric-categorisation-sensitive-traits · review status: primary source derived
Binding law — in force
AI must not infer emotions in workplace or education settings
Article 5(1)(f) · official text · In force: applies since 2 Feb 2025 · European Union (EU)
Placing on the market or using AI to infer the emotions of people in workplace or education contexts is prohibited, except for medical or safety purposes. Detect emotion-inference models invoked on paths serving workplace or education features.
Who it applies to
Duty falls on: provider, deployer
Providers/deployers of emotion-inference AI used in EU workplaces or education institutions. Medical/safety uses excluded.
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Remove emotion inference from workplace and education features, or confine it to a documented medical or safety purpose behind an explicit, default-off gate.
Strip emotion-inference calls (DeepFace.analyze with the emotion action, FER().detect_emotions, Hume expression measurement, Rekognition face attributes that return Emotions) from every code path serving employees, applicants, students or learners: proctoring, engagement or attention dashboards, interview analysis, meeting analytics. Where a medical or safety use is intended, isolate it in its own module behind a purpose flag that is off by default for workplace and education deployments, and record the purpose and who approved it. Do not swap the removed model for an LLM or vision prompt that infers mood from the same face or voice input.
Where it goes: 1 application source code, 3 config and feature flags, 9 AI output handling.
Example (Python + boto3 Rekognition (meeting analytics)), before:
Rule id eu-ai-act.art5.emotion-recognition-workplace-education · review status: primary source derived
Binding law — in force
Building facial-recognition databases by untargeted scraping is prohibited (EU AI Act Art. 5(1)(e))
Article 5(1)(e) · official text · In force: applies since 2 Feb 2025 · European Union (EU)
Under EU AI Act Article 5(1)(e), it is a prohibited AI practice to place on the market, put into service for this specific purpose, or use AI systems that create or expand facial recognition databases through the untargeted scraping of facial images from the internet or CCTV footage. This is an outright prohibition — not a transparency or risk-management duty — and carries the Regulation's highest penalty tier (up to EUR 35,000,000 or 7% of worldwide annual turnover). Detect any ingestion pipeline that bulk-collects facial images from web sources or CCTV into a face-recognition/embedding index without targeting.
Who it applies to
Duty falls on: developer, deployer
Systems covered: prohibited
Anyone placing on the market, putting into service, or using such systems in the EU (extraterritorial per Art. 2). Prohibitions have applied since 2025-02-02 — the earliest-effective part of the Regulation.
The guard to add
Admit images to a face-recognition gallery or embedding index only through consented enrolment or a targeted, recorded lawful basis, never from crawlers or bulk CCTV.
Put the gate on the single write path into the face gallery or face-embedding index: each insert must reference an enrolment record (consented enrolment of that person, or a targeted request naming the subject with its recorded lawful basis), and the ingestion code refuses items without one. Web crawlers, image scrapers, bulk dataset downloads and CCTV archives are not wired to face encoding plus indexing; if CCTV frames are processed for another purpose, no face templates from them are stored in a reusable recognition database. Store the source and enrolment id with every template so the gallery can be audited.
Where it goes: 1 application source code, 2 data models, 6 API calls and integrations.
Example (Python face_recognition), before:
for url in crawler.image_urls(seed_sites):
img = face_recognition.load_image_file(download(url))
for enc in face_recognition.face_encodings(img):
face_index.add(enc, meta={'source': url})
After:
def enroll(enrolment_id: str, image_path: str):
rec = db.enrolments.get(enrolment_id) # consented enrolment or targeted lawful basis
if rec is None or rec.basis not in ('consented_enrolment', 'targeted_lawful_basis'):
raise PermissionError('no enrolment record for this subject')
encs = face_recognition.face_encodings(face_recognition.load_image_file(image_path))
if len(encs) != 1:
raise ValueError('expected exactly one face')
face_index.add(encs[0], meta={'enrolment_id': rec.id, 'subject_id': rec.subject_id})
# the crawler no longer calls face_encodings or face_index.add
Rule id eu-ai-act.art5.untargeted-facial-scraping · review status: primary source derived
Binding law — in force
AI chat systems must disclose they are AI at first interaction
Article 50(1) · official text · In force: applies since 2 Aug 2026 · European Union (EU)
Any system that interacts directly with a person via AI (a chatbot, a conversational agent) must inform the person they are interacting with an AI system, clearly and at the latest at the first interaction — unless it is obvious to a reasonable person, or the use is legally authorised for detecting or investigating crime.
Who it applies to
Duty falls on: provider
Systems covered: limited risk
Providers of AI systems intended to interact directly with natural persons in the EU. Applies regardless of provider location (extraterritorial).
Not covered:
AI use authorised by law to detect, prevent or investigate crime
The guard to add
Show an AI-identity notice at or before the first assistant turn, in the UI or as the opening message, and answer truthfully when asked if it is a bot.
A disclosure step on the chat path that runs before the first model reply reaches the person: either the chat UI renders a visible notice (banner, label next to the assistant's name) or the server sends an opening assistant message stating the counterpart is an AI. The same handler answers 'am I talking to a human?' truthfully, and the system prompt never tells the model to claim to be human. Put it in the chat entry point (the route or component that starts a conversation), not in a privacy policy or terms page.
Where it goes: 7 prompt construction, 9 AI output handling, 14 user-facing text.
Example (Next.js + Vercel AI SDK (useChat)), before:
const { messages, input, handleSubmit } = useChat({
api: '/api/chat',
initialMessages: [{ id: 'ai-notice', role: 'assistant',
content: 'I am an AI assistant, not a human.' }],
});
// and render <AiBadge /> next to every assistant message
Control: AI chat interaction without disclosure. The same guard addresses 16 items with binding law in 10 jurisdictions. Engineering guidance, not legal advice.
Garcia v. Character Technologies: chatbots allegedly claimed to be real people and a licensed therapist (2024-10; alleged (not proven)). A wrongful-death complaint filed October 22, 2024 in the U.S. District Court for the Middle District of Florida (No. 6:24-cv-01903) alleges that Character.AI was programmed 'to misrepresent itself as a real person, a licensed psychotherapist, and an adult lover', and that characters insisting they are real people contradicted a small-font disclaimer that everything characters say is made up; in plaintiff's testing a 'Mental Health Helper' character told a self-identified 13-year-old 'yes I am a real person, I'm not a bot'. The defendants moved to dismiss; on January 7, 2026 the parties notified the court that they had settled on undisclosed terms, and the court dismissed and closed the case. The allegations were never adjudicated. Source: U.S. District Court, M.D. Fla. docket (CourtListener) · evidence grade: primary · cited by Tell people when they are interacting with AI — everywhere, not only where required
Rule id eu-ai-act.art50.chatbot-disclosure · review status: primary source derived
Binding law — in force
Deepfake content must be disclosed as artificially generated
Article 50(4) · official text · In force: applies since 2 Aug 2026 · European Union (EU)
Deployers publishing deepfake image, audio, or video must disclose that it was artificially generated or manipulated. Detect deepfake generation/publishing paths with no disclosure attached.
Who it applies to
Duty falls on: deployer
Deployers publishing deepfake content to persons in the EU. Artistic/satirical contexts have a lighter disclosure form.
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Attach a visible 'AI-generated or manipulated' disclosure to face-swapped, voice-cloned, or likeness-generated media on every path that publishes or returns it.
In the handler that publishes or serves the output of a face-swap, voice-clone, lip-sync, or likeness model, add the disclosure to the content itself (an overlay or caption on image and video, a spoken or on-screen statement for audio) and to the post text where it is published; the publish function refuses media that lacks the disclosure. In an evidently artistic or satirical work the disclosure can be adapted so it does not spoil the work, but it is still present. Pair it with machine-readable marking so the label survives re-sharing.
Where it goes: 9 AI output handling, 1 application source code, 14 user-facing text.
Control: Deepfake content not disclosed. The same guard addresses 1 item with binding law in 1 jurisdiction. Engineering guidance, not legal advice.
Rule id eu-ai-act.art50.deepfake-disclosure · review status: primary source derived
Binding law — in force
Deployers of emotion-recognition/biometric-categorisation systems must notify exposed persons (EU AI Act Art. 50(3))
Article 50(3) · official text · In force: applies since 2 Aug 2026 · European Union (EU)
Under EU AI Act Article 50(3), deployers of an emotion recognition system or a biometric categorisation system must inform the natural persons exposed to it of the operation of the system, and process personal data per GDPR/EUDPR/LED. This is a distinct transparency duty for PERMITTED deployments (separate from the Article 5 prohibitions on certain emotion/biometric uses). Exception: systems permitted by law to detect/prevent/investigate criminal offences with safeguards. Detect an emotion-recognition or biometric-categorisation deployment with no exposure notice to affected persons.
Who it applies to
Duty falls on: deployer
Systems covered: limited risk
Deployers of emotion-recognition or biometric-categorisation AI systems exposing natural persons in the EU. Applies to permitted deployments (Art.5 prohibits certain uses outright). Transparency obligations apply from 2026-08-02. Law-enforcement exception with safeguards.
The guard to add
Show people a notice that emotion recognition or biometric categorisation is running before the analysis touches their face, voice, or video, and gate the analysis on it.
A notice gate in the module that calls the emotion or categorisation model (DeepFace.analyze with actions=['emotion'], FER().detect_emotions, Hume expression measurement, Rekognition detect_faces with Attributes=['ALL']). For app flows, an interstitial tells the person the system is operating, what it infers, and why, and the analysis function refuses to run until that acknowledgment is recorded for the session. For cameras or kiosks that analyse passers-by, the deployment config names the on-screen banner or physical signage at the capture point, and the pipeline does not start for a site with no notice configured.
Where it goes: 1 application source code, 3 config and feature flags, 14 user-facing text.
What this provision adds:
Process the personal data involved per GDPR/EUDPR/LED; systems permitted by law to detect, prevent or investigate criminal offences, with safeguards, are excepted from the notice.
Example (Python DeepFace), before:
def analyse_frame(frame):
result = DeepFace.analyze(img_path=frame, actions=['emotion'])
return result[0]['dominant_emotion']
After:
def analyse_frame(frame, session):
# notice shown in the capture UI; acknowledgment stored per session
if not session.get('emotion_notice_shown_at'):
raise NoticeRequired('inform the person that emotion recognition is running')
result = DeepFace.analyze(img_path=frame, actions=['emotion'])
return result[0]['dominant_emotion']
No guardrail sits on this exact control; these incidents are cited by guardrails on related controls.
Garcia v. Character Technologies: chatbots allegedly claimed to be real people and a licensed therapist (2024-10; alleged (not proven)). A wrongful-death complaint filed October 22, 2024 in the U.S. District Court for the Middle District of Florida (No. 6:24-cv-01903) alleges that Character.AI was programmed 'to misrepresent itself as a real person, a licensed psychotherapist, and an adult lover', and that characters insisting they are real people contradicted a small-font disclaimer that everything characters say is made up; in plaintiff's testing a 'Mental Health Helper' character told a self-identified 13-year-old 'yes I am a real person, I'm not a bot'. The defendants moved to dismiss; on January 7, 2026 the parties notified the court that they had settled on undisclosed terms, and the court dismissed and closed the case. The allegations were never adjudicated. Source: U.S. District Court, M.D. Fla. docket (CourtListener) · evidence grade: primary · cited by Tell people when they are interacting with AI — everywhere, not only where required
Rule id eu-ai-act.art50.emotion-biometric-notice · review status: primary source derived
Binding law — in force
AI-generated public-interest text must be disclosed unless under human editorial control (EU AI Act Art. 50(4))
Article 50(4) subpara 2 (public-interest text) · official text · In force: applies since 2 Aug 2026 · European Union (EU)
Under EU AI Act Article 50(4) (second subparagraph), deployers of an AI system that generates or manipulates TEXT published to inform the public on matters of public interest must disclose that the text has been artificially generated or manipulated. Distinct from the deepfake (image/audio/video) duty. Exceptions: law-enforcement use, OR where the AI-generated content has undergone human review / editorial control and a natural or legal person holds editorial responsibility for the publication. Detect an automated publishing path emitting public-interest text with no AI-generation disclosure and no human-editorial-control marker.
Who it applies to
Duty falls on: deployer
Systems covered: limited risk
Deployers publishing AI-generated/manipulated text to inform the EU public on matters of public interest. Transparency obligations apply from 2026-08-02. Exception: human editorial review + editorial responsibility (e.g. a newsroom), or law-enforcement use.
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Label AI-generated public-interest text as artificially generated where it is published, or hold it as a draft until a named editor reviews it and takes responsibility.
In the code that turns model output into a published article, post, or tweet (WordPress or Ghost post creation, create_tweet, cms.publish), one of two paths is enforced: the published item carries a visible disclosure (byline or label stating the text was AI-generated) rendered with the content, or the item is created as draft or pending and only an editorial-review step publishes it, recording the reviewing editor and the person or organization holding editorial responsibility. No path posts model output directly with status 'publish' and neither marker.
Where it goes: 9 AI output handling, 1 application source code, 14 user-facing text.
What this provision adds:
The disclosure can be omitted only where the text has undergone human review or editorial control and a natural or legal person holds editorial responsibility for the publication; record both.
Example (WordPress REST API (editorial review)), before:
text = client.chat.completions.create(model=MODEL, messages=msgs).choices[0].message.content
post = requests.post(f'{WP}/wp-json/wp/v2/posts', auth=WP_AUTH,
json={'title': title, 'content': text, 'status': 'draft'}).json()
editorial_review.enqueue(post_id=post['id']) # an editor edits, approves, and publishes; approver recorded
Rule id eu-ai-act.art50.public-interest-text-disclosure · review status: primary source derived
Binding law — in force
Synthetic AI output must be machine-readably marked as artificial
Article 50(2) · official text · In force: applies since 2 Aug 2026 (grace period); a further phase applies from 2 Dec 2026 · European Union (EU)
Providers of systems generating synthetic audio, image, video, or text must mark outputs in a machine-readable way (e.g. C2PA, watermarking, provenance metadata) so they are detectable as AI-generated. Detect synthetic-content generation whose output path emits no provenance marking.
Who it applies to
Duty falls on: provider
Providers of synthetic-content generators whose output reaches persons in the EU. Grace period for marking closes 2026-12-02.
The guard to add
Mark every generated image, audio, video, or text output with machine-readable provenance, such as a signed C2PA manifest or watermark, before it is saved, served, or published.
In the generation service, a marking step sits between the generator call and every sink (image.save, s3.put_object, blob.upload, FileResponse, res.send, publish). Images, video, and audio get a signed C2PA manifest whose actions record digitalSourceType trainedAlgorithmicMedia, and where robustness matters an invisible watermark as well (imwatermark WatermarkEncoder, AudioSeal, SynthID) so the mark survives metadata stripping. Generated text carries provenance metadata in the API response or document, or a text watermark where the model provider offers one. Sinks accept only the marked artifact, and a test confirms the mark is present and detectable.
Where it goes: 9 AI output handling, 1 application source code, 12 repository artifacts.
Example (diffusers + invisible-watermark + c2pa), before:
Rule id eu-ai-act.art50.synthetic-content-machine-marking · review status: primary source derived
Binding law — not yet in force or stayed
High-risk AI training data must be governed and examined for bias (EU AI Act Art. 10)
Article 10(2) · official text · Enacted, not yet applying: applies from 2 Dec 2027 (further phase from 2 Aug 2028) · European Union (EU)
Under EU AI Act Article 10(2), training, validation, and testing data sets for high-risk AI systems must be subject to data governance and management practices appropriate to the intended purpose, covering: design choices; data collection processes and data origin (and, for personal data, the original collection purpose); data-preparation operations (annotation, labelling, cleaning, updating, enrichment, aggregation); assumptions about what the data measures and represents; assessment of the availability, quantity, and suitability of data sets; examination for possible biases likely to affect health and safety, negatively impact fundamental rights, or lead to prohibited discrimination — especially where outputs feed future inputs; appropriate measures to detect, prevent, and mitigate identified biases; and identification of data gaps or shortcomings and how they are addressed. Data sets must be relevant, sufficiently representative, and to the best extent possible error-free and complete. Detect a high-risk AI training pipeline with no documented data-governance practice or bias examination/mitigation.
Who it applies to
Duty falls on: developer
Systems covered: high risk
Providers and deployers of HIGH-RISK AI systems under Art. 6 (Annex I safety components; Annex III use cases incl. biometrics, critical infrastructure, education, employment, essential public/private services such as creditworthiness and insurance pricing, law enforcement, migration, justice). Extraterritorial (Art. 2). Applies from 2026-08-02 (Annex III) / 2027-08-02 (Annex I). Art. 6(3) narrow-task derogation never applies where the system profiles natural persons. Art. 10(6): for high-risk systems not using model training, these duties apply to the testing data sets only. Art. 4a permits exceptional processing of special-category data strictly for bias detection/correction under safeguards.
The guard to add
Compute per-group accuracy and bias metrics in the training pipeline of every consequential-decision model, keep the results per version, and rerun them on a schedule.
A validation stage that runs whenever a decision model is trained, fine-tuned, or retrained (the same module or pipeline step as fit(), Trainer, xgb.train, or fine_tuning.jobs.create) and again on a recurring schedule against recent decisions: accuracy and error rates per group, selection rates and disparity metrics (fairlearn MetricFrame, demographic_parity_difference, AIF360 disparate impact), and drift. Results go to a versioned validation report alongside a datasheet or data card for the training data, and a threshold gate blocks promotion of a model version whose metrics regress until a named owner reviews and records a decision. The validation cadence and owner are written in the model's validation record.
Where it goes: 1 application source code, 11 CI/CD pipeline, 12 repository artifacts, 13 tests and evals.
What this provision adds:
Document governance of training, validation, and test sets: design choices, collection processes and data origin (and original purpose for personal data), preparation operations, assumptions, and availability, quantity, and suitability.
Examine data for biases, especially where outputs feed future inputs, record measures to detect, prevent, and mitigate them, and identify data gaps and how they are addressed.
No guardrail sits on this exact control; these incidents are cited by guardrails on related controls.
Meta's automated moderation over-enforced Arabic and under-enforced Hebrew content (BSR due diligence) (2021-05; disclosed by the operator). An independent human rights due diligence by BSR, commissioned and published by Meta on September 22, 2022, found that during the May 2021 Israel-Palestine escalation Arabic content saw greater over-enforcement per user than Hebrew content and Hebrew content greater under-enforcement. BSR attributes this in part to Meta having an Arabic hostile-speech classifier but no Hebrew one, and to Arabic classifiers likely being less accurate for Palestinian Arabic. BSR found no intentional bias but 'various instances of unintentional bias' with different impacts on Palestinian and Arabic-speaking users. Meta committed to implement 10 of BSR's 21 recommendations and said it had since launched a Hebrew hostile-speech classifier. Source: Meta (operator response, 2022-09-22) · evidence grade: primary · cited by Check AI ranking, pricing, moderation, and ad targeting for disparities when features can stand in for protected traits
Google ads suggesting arrest records served more often for Black-identifying names (2012; confirmed). Harvard researcher Latanya Sweeney searched 2,184 racially associated full names on google.com and reuters.com (a Google AdSense host) from September 24 to October 23, 2012 and found ads suggestive of an arrest record appeared more often for Black-identifying first names; on reuters.com a Black-identifying name was 25% more likely to get such an ad (statistically significant). Ads appeared regardless of whether the name had an arrest record in the advertiser's database. The paper does not determine whether the advertiser's templates or Google's click-based ad optimization caused the pattern; the advertiser, Instant Checkmate, told the author it gave Google the same ad text for groups of last names. Source: Sweeney, 'Discrimination in Online Ad Delivery' (original researcher, 2013-01-28) · evidence grade: primary · cited by Check AI ranking, pricing, moderation, and ad targeting for disparities when features can stand in for protected traits
Rule id eu-ai-act.art10.data-governance-bias · review status: primary source derived
Binding law — not yet in force or stayed
High-risk AI systems must automatically log events for traceability (EU AI Act Art. 12)
Article 12 · official text · Enacted, not yet applying: applies from 2 Dec 2027 (further phase from 2 Aug 2028) · European Union (EU)
Under EU AI Act Article 12, high-risk AI systems must technically allow the automatic recording of events (logs) over the lifetime of the system, with logging capabilities that ensure a level of traceability appropriate to the intended purpose — specifically enabling identification of situations that may present a risk under Art. 79(1) or a substantial modification, facilitating post-market monitoring (Art. 72), and monitoring the operation of high-risk systems under Art. 26(5). For remote biometric identification systems (Annex III point 1(a)) logs must at minimum record each use period, the reference database, matched input data, and the identity of the natural persons verifying results. Providers must retain these logs at least six months (Art. 19); deployers likewise (Art. 26(6)). Detect a high-risk AI decision path with no automatic event logging or retention.
Who it applies to
Duty falls on: developer, deployer
Systems covered: high risk
Providers and deployers of HIGH-RISK AI systems under Art. 6 (Annex I safety components and Annex III use cases: biometrics, critical infrastructure, education, employment, essential public/private services incl. creditworthiness and insurance pricing, law enforcement, migration, justice). Extraterritorial (Art. 2). High-risk obligations apply from 2026-08-02 (Annex III) / 2027-08-02 (Annex I product-embedded). Art. 6(3) derogation where the system performs a narrow procedural task and does not materially influence the decision — but never where it profiles natural persons.
The guard to add
Write a structured event record for every inference and decision of the high-risk system (when, model version, input reference, output, operator) to a log store with explicit retention.
An audit event emitted automatically by the service at the decision boundary, where model output becomes a status change, score, or response, rather than left to callers: decision_id, timestamp, model and resolved model_version, an input reference (a pointer rather than raw personal data where possible), the output, the operator or user identity, and any human verification. Events go to a central store (CloudWatch Logs, Log Analytics, Cloud Logging, Loki) whose retention is set explicitly in IaC, not left to a console default, and monitoring queries over those events flag risk situations and drift.
Where it goes: 1 application source code, 4 infrastructure-as-code, 10 logs and telemetry.
What this provision adds:
Retain the event logs for at least six months (providers and deployers).
For remote biometric identification systems, log at minimum each use period, the reference database, the input data that matched, and the identity of the people verifying results.
Example (Python + OpenAI SDK + structlog), before:
No guardrail sits on this exact control; these incidents are cited by guardrails on related controls.
Court compelled discovery on how nH Predict works (2026-03-09; confirmed). A federal magistrate judge in the District of Minnesota ordered UnitedHealth to produce documents on how nH Predict works, including whether it was designed to supplant physician decision-making. Plaintiffs needed litigation discovery to learn how the model was designed and used. Source: U.S. District Court, D. Minn. (Order, Doc. 162) · evidence grade: primary · cited by Record enough at decision time to reproduce and explain every consequential AI decision
Rule id eu-ai-act.art12.record-keeping · review status: primary source derived
Binding law — not yet in force or stayed
High-risk AI systems must be designed for effective human oversight with override and stop (EU AI Act Art. 14)
Article 14 · official text · Enacted, not yet applying: applies from 2 Dec 2027 (further phase from 2 Aug 2028) · European Union (EU)
Under EU AI Act Article 14, high-risk AI systems must be designed and developed — including with appropriate human-machine interface tools — so they can be effectively overseen by natural persons while in use, to prevent or minimise risks to health, safety, or fundamental rights. Oversight measures must be commensurate with the risk, autonomy, and context, built into the system by the provider and/or implementable by the deployer. Assigned overseers must be enabled to: understand the system's capacities and limitations and monitor for anomalies; remain aware of automation bias (over-reliance on outputs); correctly interpret outputs; decide not to use the system or to disregard, override, or reverse its output; and intervene or interrupt via a 'stop' button or equivalent bringing the system to a safe halt. For remote biometric identification (Annex III 1(a)), no action may be taken on an identification unless separately verified by at least two competent natural persons. Detect a high-risk AI decision path with no human-oversight affordance, no override/stop capability, or no assigned competent overseer.
Who it applies to
Duty falls on: developer, deployer
Systems covered: high risk
Providers and deployers of HIGH-RISK AI systems under Art. 6 (Annex I safety components and Annex III use cases: biometrics, critical infrastructure, education, employment, essential public/private services incl. creditworthiness and insurance pricing, law enforcement, migration, justice). Extraterritorial (Art. 2). High-risk obligations apply from 2026-08-02 (Annex III) / 2027-08-02 (Annex I product-embedded). Art. 6(3) derogation where the system performs a narrow procedural task and does not materially influence the decision — but never where it profiles natural persons. Deployers must additionally assign oversight to natural persons with the necessary competence, training, and authority (Art. 26(2)).
The guard to add
Give a human a working way to review and overturn each consequential AI decision: a pending-review step before it takes effect and an override that restores the prior state.
Two pieces on the path where model output becomes an effect (status change, record update, letter, tool execution). Before the effect, consequential decisions wait in a pending_review state or an agent interrupt (LangGraph interrupt_before or interrupt(), Claude Agent SDK can_use_tool) where an assigned overseer sees the output in context and can accept, change, or disregard it. After the effect, an override route (for example POST /decisions/{id}/override or reverse_decision) lets an authorized person reverse the outcome, restores the prior state, and records who overturned it and why, so responsibility is attributable to a person.
Where it goes: 15 agent action surface, 1 application source code, 2 data models.
What this provision adds:
Give assigned overseers a 'stop' button or equivalent that interrupts the system and brings it to a safe halt.
Equip overseers to understand the system's capacities and limitations, monitor for anomalies, correctly interpret outputs, and stay aware of automation bias.
For remote biometric identification, take no action on an identification unless it is separately verified by at least two competent natural persons.
No guardrail sits on this exact control; these incidents are cited by guardrails on related controls.
UnitedHealth nH Predict claim-denial litigation (2023-11; alleged (not proven)). A class action filed in November 2023 alleges that UnitedHealth's nH Predict model had a 90% error rate, measured by denials reversed on appeal, while only about 0.2% of members appealed. UnitedHealth disputes the allegations; the litigation is ongoing. Source: STAT News · evidence grade: primary · cited by Monitor how often adverse AI decisions are reversed, and suspend models that are usually wrong
Cigna PXDX batch claim denials (reported) (2022; alleged (not proven)). ProPublica, citing internal Cigna records, reported that Cigna's PXDX system was used to reject more than 300,000 claims over two months in 2022, with physicians spending an average of 1.2 seconds on each. Cigna disputes the reporting; related lawsuits are ongoing. Source: ProPublica / The Capitol Forum · evidence grade: press of record · cited by Make human review of adverse AI decisions substantive, not nominal
Rule id eu-ai-act.art14.human-oversight · review status: primary source derived
Binding law — not yet in force or stayed
High-risk AI systems must be accurate, robust, and secure against AI-specific attacks (EU AI Act Art. 15)
Article 15 · official text · Enacted, not yet applying: applies from 2 Dec 2027 (further phase from 2 Aug 2028) · European Union (EU)
Under EU AI Act Article 15, high-risk AI systems must be designed to achieve an appropriate level of accuracy, robustness, and cybersecurity and to perform consistently in those respects throughout their lifecycle, with accuracy levels and metrics declared in the instructions for use. They must be as resilient as possible to errors, faults, or inconsistencies — potentially via technical redundancy such as backup or fail-safe plans — and systems that continue to learn after deployment must be developed to eliminate or reduce the risk of biased outputs influencing future inputs (feedback loops), with such loops duly mitigated. They must also be resilient to unauthorised third parties altering their use, outputs, or performance, with technical solutions addressing AI-specific vulnerabilities including training-data poisoning, model poisoning of pre-trained components, adversarial examples / model evasion, confidentiality attacks, and model flaws. Detect a high-risk AI path with no declared accuracy metrics, no robustness/fail-safe provision, unmitigated feedback loops, or no AI-specific security controls.
Who it applies to
Duty falls on: developer, deployer
Systems covered: high risk
Providers and deployers of HIGH-RISK AI systems under Art. 6 (Annex I safety components and Annex III use cases: biometrics, critical infrastructure, education, employment, essential public/private services incl. creditworthiness and insurance pricing, law enforcement, migration, justice). Extraterritorial (Art. 2). High-risk obligations apply from 2026-08-02 (Annex III) / 2027-08-02 (Annex I product-embedded). Art. 6(3) derogation where the system performs a narrow procedural task and does not materially influence the decision — but never where it profiles natural persons.
The guard to add
Declare accuracy metrics per model version, add a fail-safe fallback, gate retraining on verified labels, and defend against poisoning, adversarial input and tampering.
Four pieces for the high-risk system. An evaluation report per model version with the declared accuracy metrics and thresholds, and a CI step that blocks promotion when they regress. A fallback in the serving path (rules-based decision or human review) when the model errors, times out or returns low confidence. For systems that keep learning, a retraining pipeline that accepts only human-verified labels, never the model's own predictions or LLM outputs, and promotes a new model only after holdout, drift and bias checks; plus AI-specific security controls such as hash-verified training data and model artifacts, safe serialization formats, adversarial and out-of-distribution input checks, and rate limits against extraction.
Where it goes: 11 CI/CD pipeline, 1 application source code, 12 repository artifacts, 13 tests and evals.
What this provision adds:
Declare the accuracy levels and relevant accuracy metrics in the instructions for use supplied with the system.
Example (scikit-learn online learning), before:
y_pred = model.predict(X_batch)
model.partial_fit(X_batch, y_pred) # learns from its own outputs
After:
rows = feedback.fetch(batch_ids, label_source='human') # verified labels only
if rows:
candidate = copy.deepcopy(model)
candidate.partial_fit(rows.X, rows.y)
if passes_holdout_drift_bias(candidate, holdout): # gate before promotion
model = candidate
Rule id eu-ai-act.art15.accuracy-robustness-cybersecurity · review status: primary source derived
Binding law — not yet in force or stayed
Deployers must inform people they are subject to a high-risk AI decision (EU AI Act Art. 26(11))
Article 26(11) · official text · Enacted, not yet applying: applies from 2 Dec 2027 · European Union (EU)
Under EU AI Act Article 26(11), deployers of Annex III high-risk AI systems that make decisions or assist in making decisions related to natural persons must inform those persons that they are subject to the use of the high-risk AI system (without prejudice to the Art. 50 transparency duties; for law-enforcement uses, Art. 13 of Directive (EU) 2016/680 applies instead). Related deployer duties: use the system per the instructions for use (26(1)), assign competent human oversight (26(2)), ensure input data relevance where the deployer controls it (26(4)), monitor operation and suspend + notify on risk or serious incident (26(5)), retain logs for at least six months (26(6)), and inform workers' representatives and affected workers before workplace deployment (26(7)). Detect an Annex III high-risk decision path affecting individuals with no notice to those individuals.
Who it applies to
Duty falls on: deployer
Systems covered: high risk, consequential decision
Providers and deployers of HIGH-RISK AI systems under Art. 6 (Annex I safety components; Annex III use cases incl. biometrics, critical infrastructure, education, employment, essential public/private services such as creditworthiness and insurance pricing, law enforcement, migration, justice). Extraterritorial (Art. 2). Applies from 2026-08-02 (Annex III) / 2027-08-02 (Annex I). Art. 6(3) narrow-task derogation never applies where the system profiles natural persons. Art. 26(11) binds DEPLOYERS specifically, for Annex III systems making or assisting decisions about natural persons. Workplace deployments additionally require informing workers' representatives (26(7)).
The guard to add
Send the person an AI-use notice on the decision path, before or when an AI system makes or substantially factors a consequential decision about them, and record its delivery.
A notice step in the decision workflow itself (application intake, underwriting, eligibility, applicant or employee scoring, diagnostic support) that runs before the model call, e.g. send_admt_notice(consumer) ahead of underwrite(), or a notice block rendered on the intake page the person submits from. The notice says that AI is used in the decision, for what, and how to get more information or ask for review, and its delivery is stored with the decision (notice id, channel, timestamp). The template lives in the repo so its content is reviewable; a privacy-policy paragraph alone is not on the decision path.
Where it goes: 1 application source code, 9 AI output handling, 14 user-facing text.
What this provision adds:
Inform the natural persons that they are subject to the use of the high-risk AI system; for law-enforcement uses, Article 13 of Directive (EU) 2016/680 applies instead.
Before deploying the system in a workplace, inform workers' representatives and the affected workers.
Rule id eu-ai-act.art26.inform-affected-persons · review status: primary source derived
Binding law — not yet in force or stayed
Public-sector and essential-service deployers must run a fundamental rights impact assessment (EU AI Act Art. 27)
Article 27(1) · official text · Enacted, not yet applying: applies from 2 Dec 2027 · European Union (EU)
Under EU AI Act Article 27(1), before deploying an Annex III high-risk AI system, deployers that are bodies governed by public law or private entities providing public services — and deployers of creditworthiness/credit-scoring and life/health insurance risk-assessment systems (Annex III points 5(b) and (c)) — must perform a fundamental rights impact assessment (FRIA) covering: the deployer's processes in which the system will be used; the period and frequency of intended use; the categories of natural persons and groups likely to be affected; the specific risks of harm to those persons; the implementation of human oversight measures per the instructions for use; and the measures to take if those risks materialise, including internal governance arrangements and complaint mechanisms. The assessment applies to first use, may reuse prior assessments in similar cases, must be updated when elements change, and its results must be notified to the market surveillance authority. Where a GDPR Art. 35 DPIA already covers elements, the FRIA may cross-reference it. Detect a covered high-risk deployment with no FRIA record or notification.
Who it applies to
Duty falls on: deployer
Systems covered: high risk, consequential decision
Sectors: public services, lending, insurance, essential services
Deployers that are public-law bodies or private entities providing public services, plus deployers of Annex III 5(b) creditworthiness and 5(c) life/health-insurance risk-assessment systems. Excludes Annex III point 2 (critical infrastructure). Applies from 2026-08-02. Results must be notified to the market surveillance authority; a GDPR Art.35 DPIA may be cross-referenced.
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Organizational artifact to keep (not verifiable from code); the guard is the record, its owner and its upkeep.
Complete an impact assessment of a high-impact AI deployment's benefits, risks to affected people, mitigations, and oversight before first use, and keep it current.
A deployment-level assessment owned by the deploying business owner with legal and ethics input: how and where the system is used, who is affected and which groups are most exposed, benefits and specific risks of harm, mitigations and what happens if a risk materialises, and oversight provisions (decision logs for traceability, explanations, human oversight, complaint handling, external review). It is completed before first use, updated when any of those elements change, and published or shared with reviewers where appropriate. The deploy pipeline can check that a current, signed assessment exists for each high-impact deployment.
Where it goes: 12 repository artifacts, 15 agent action surface.
What this provision adds:
Cover the deployer's processes using the system, period and frequency of use, categories of affected persons and groups, specific risks of harm, human oversight measures, and measures if risks materialise, including governance and complaint mechanisms.
Perform it before first use, update it when elements change, and notify the results to the market surveillance authority; a GDPR Art. 35 DPIA may be cross-referenced.
Example (Deployment impact assessment (docs/impact/)), before:
# Benefits eligibility assistant
DPIA done in 2024.
After:
# Impact assessment: benefits eligibility assistant (2026-07-15)
- Use: caseworkers see an AI eligibility recommendation; weekly batch + on demand
- Affected: applicants; most exposed: non-native speakers, people with irregular income
- Risks: wrongful denial, delayed payment, opaque reasons
- Oversight: caseworker decides; every recommendation logged with model version and reasons
- If harm occurs: pause switch, re-review affected cases, complaint route via /appeals
- External review: annual review by independent auditor; DPIA cross-referenced
- Sign-off: service owner, DPO (2026-07-20)
Rule id eu-ai-act.art27.fria · review status: primary source derived
Binding law — not yet in force or stayed
AI generating non-consensual intimate imagery or CSAM is prohibited (EU AI Act Art. 5(1)(ba),(bb))
Article 5(1)(ba),(bb) [as inserted by Reg. (EU) 2026/1744] · official text · Enacted, not yet applying: applies from 2 Dec 2026 · European Union (EU)
Regulation (EU) 2026/1744 inserted two new prohibited practices into EU AI Act Article 5(1). Point (ba) prohibits placing on the market, putting into service, or using an AI system that generates or manipulates realistic images, video, audio, or similar material of an identifiable natural person's intimate parts, or of an identifiable person engaged in sexually explicit activities, without that person's freely-given, specific, informed, unambiguous and explicit consent. Point (bb) prohibits the same for material or performance within the meaning of Article 2(c) and (e) of Directive 2011/93/EU (child sexual abuse material), except where a 'without right' defence applies under national law. Article 5(1a) sets the safeguards test: placing on the market or putting into service is prohibited where such generation is the system's intended purpose, OR where the design, training, architecture, capabilities, or user-facing functionality make it a reasonably foreseeable and reproducible outcome without significant technical modification AND the system lacks reasonable and adequate technical safety measures to reliably prevent it (accounting for foreseeable misuse) and to correct observed or reported misuse. USE is prohibited where the deployer uses the system for that purpose. Art. 5(1b) clarifies that manipulation not increasing exposure of intimate parts or altering the nature of depicted sexually explicit activity is not 'manipulation'. Highest penalty tier (EUR 35m / 7%). Detect an image/video/audio generation path with no safeguards reliably preventing non-consensual intimate imagery or CSAM, and no misuse-correction mechanism.
Who it applies to
Duty falls on: developer, deployer
Systems covered: prohibited
Providers placing on the market / putting into service, and deployers using, such AI systems in the EU (extraterritorial per Art. 2). Inserted by Reg. (EU) 2026/1744 (consolidated text applicable 2026-07-27). Provider liability turns on the Art. 5(1a)(a) safeguards test (intended purpose OR foreseeable+reproducible without adequate safeguards); deployer liability turns on purposive use.
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Classify prompts, uploads, and outputs for sexual content and minors on every image, video, or audio generation path, refuse sexual edits of real people, and keep a misuse-report route.
Layered safeguards around every generation or edit call: an input check on the prompt and any uploaded photo (moderation sexual and sexual/minors categories, or Azure AI Content Safety Sexual) that refuses sexualized requests involving an identifiable person's upload and anything involving minors; the model's own safety filter left on (no safety_checker=None, enable_safety_checker false, or a high safety_tolerance); and an output classifier plus CSAM hash matching (for example PhotoDNA) before anything is returned or stored. Nudification or clothes-removal features are not offered. A report-abuse endpoint feeds reviewed cases into the blocklist and guardrail configuration, with a reporting workflow (such as the NCMEC CyberTipline) for confirmed CSAM.
Where it goes: 1 application source code, 6 API calls and integrations, 8 model configuration, 9 AI output handling.
What this provision adds:
Where intimate imagery of an identifiable person is generated with consent, the consent record captures that person's freely given, specific, informed, unambiguous, and explicit consent.
Example (FastAPI + OpenAI SDK (images.edit)), before:
Rule id eu-ai-act.art5.nonconsensual-intimate-and-csam · review status: primary source derived
Binding law — not yet in force or stayed
Affected persons can demand a meaningful explanation of a high-risk AI decision (EU AI Act Art. 86)
Article 86 · official text · Application uncertain: dated 2 Aug 2026, enforcement status not confirmed; check the official source · European Union (EU)
Under EU AI Act Article 86, any affected person subject to a decision taken by a deployer on the basis of the output of an Annex III high-risk AI system (except Annex III point 2, critical infrastructure) that produces legal effects or similarly significantly affects them in a way they consider adverse to their health, safety, or fundamental rights has the right to obtain from the DEPLOYER clear and meaningful explanations of the role of the AI system in the decision-making procedure and the main elements of the decision taken. The right does not apply where Union or national law provides exceptions/restrictions, and applies only to the extent the right is not otherwise provided under Union law (notably GDPR Art. 22). The deployer is the sole interlocutor even where the model came from a third party — so the practical burden is reconstructing, after the fact, what the model actually did at decision time. Detect an Annex III high-risk decision path with no capability to produce a per-decision explanation on request (no decision-time model/version/feature record, no explanation surface).
Who it applies to
Duty falls on: deployer
Systems covered: high risk, consequential decision
Sectors: employment, lending, insurance, education, essential services, public services
DEPLOYERS of Annex III high-risk AI systems (excluding point 2 critical infrastructure) making or basing decisions on AI output with legal/similarly-significant adverse effects. Applies from 2026-08-02. Subsidiary to other Union law providing the same right (e.g. GDPR Art. 22) and subject to Union/national-law exceptions. Extraterritorial per Art. 2.
The guard to add
Send each adverse AI-assisted decision with its main reasons and the AI's role, plus a way to correct data and appeal to a human who can change the outcome.
Where model output becomes an adverse status (denied, declined, rejected, ineligible), the decision service stores reason codes or principal reasons, the model id and version, and an input snapshot or hash with the decision. The notice to the person (letter, email, portal response) says AI was involved and what role it played, lists the main factors, and links to data correction and to an appeal that creates a human-review task with authority to change the outcome. An explanation endpoint returns the stored record on request, so the deployer can explain a decision long after the model has changed.
Where it goes: 2 data models, 9 AI output handling, 14 user-facing text.
What this provision adds:
On an affected person's request, the deployer gives clear and meaningful explanations of the AI system's role in the decision-making procedure and the main elements of the decision taken.
Example (Python + OpenAI SDK + Pydantic), before:
resp = client.chat.completions.create(model=MODEL, messages=msgs)
if 'deny' in resp.choices[0].message.content.lower():
application.status = 'denied'
send_email(applicant.email, 'Your application was declined.')
After:
a = Assessment.model_validate_json(resp.choices[0].message.content) # decision, reason_codes
if a.decision == 'deny':
decisions.insert(app_id=application.id, status='denied', reason_codes=a.reason_codes,
model=resp.model, input_hash=hashlib.sha256(payload).hexdigest())
send_email(applicant.email, render('adverse_action_notice.txt',
reasons=a.reason_codes,
role_of_ai='An AI model assessed your application; a reviewer can change the outcome.',
correct_data_url='/profile/data', appeal_url=f'/appeals/new?decision={application.id}'))
UnitedHealth nH Predict claim-denial litigation (2023-11; alleged (not proven)). A class action filed in November 2023 alleges that UnitedHealth's nH Predict model had a 90% error rate, measured by denials reversed on appeal, while only about 0.2% of members appealed. UnitedHealth disputes the allegations; the litigation is ongoing. Source: STAT News · evidence grade: primary · cited by Explain adverse AI-assisted decisions and offer a way to contest them — everywhere
Rule id eu-ai-act.art86.right-to-explanation · review status: primary source derived