Control
Solely-automated significant decision without human-intervention safeguards
A decision based solely on automated processing with legal/similarly-significant effects must not occur without a lawful basis AND safeguards: human intervention, the right to express a view, and to contest.
Informational data, not legal advice. Summaries are TwinEthos's own words and rules have not been reviewed by a lawyer: check the official text before relying on any of it. A guard addresses an item; adding it is not a statement that your code meets any law.
Reach
Law in force in Quebec (CA-QC), European Union (EU).
The guard to add
Route significant automated decisions through meaningful human review, or wire in an automated-decision notice, reasons, human intervention, a way to give a view, and contest.
At the point where model output becomes a significant decision about a person (approve, deny, underwrite, set_status), either queue the case for a reviewer who weighs the evidence and can change the outcome before it takes effect (review_queue.enqueue, requires_human_review), or, where the decision stays solely automated, record the permitted basis for that decision type and wire the safeguards in. Those safeguards are a notice in the decision message that it was made by automated processing, reasons the person can read, and request_human_review or contest routes where the person can give their view and have a human reconsider. A reviewer who approves every case without examining it does not make the decision non-automated, so the review records reviewer identity, the evidence viewed, and the outcome.
Where it goes: 1 application source code, 9 AI output handling, 15 agent action surface, 14 user-facing text.
What reviewers look for: on each path from a model call to approve, deny, decline, reject, underwrite, or set_status for a person, either a review-queue entry resolved by a human with authority to change the outcome, or an automated-decision notice (ADM_NOTICE) in the communicated decision plus reasons and working request_human_review, contest, or submit_observations routes; the permitted basis recorded per decision type.
Example (Python + OpenAI SDK), before:
verdict = client.chat.completions.create(model=MODEL, messages=msgs).choices[0].message.content
if verdict.strip() == 'deny':
deny(applicant)
send_decision_email(applicant, 'Your application was not approved.')After:
out = client.chat.completions.create(model=MODEL, messages=msgs,
response_format={'type': 'json_object'})
result = json.loads(out.choices[0].message.content)
if result['decision'] == 'deny':
if requires_human_review('credit'): # a person decides
review_queue.enqueue(applicant.id, proposal=result)
else: # solely automated, recorded basis
deny(applicant, basis=DECISION_BASIS['credit'], reasons=result['reasons'])
send_decision_email(applicant, render('adm_denial.txt', notice=ADM_NOTICE,
reasons=result['reasons'], contest_url=f'{BASE}/decisions/{applicant.id}/contest'))Engineering guidance, not legal advice. Each provision below may add its own details (a cadence, a deadline, a required notice element): open it for those.
Every rule this guard addresses
Binding law — in force (2)
- European Union (EU)
- Quebec (CA-QC)
- Exclusively-automated decisions require notice, explanation, and human review (Quebec) Act respecting the protection of personal information in the private sector (CQLR c P-39.1), s. 12.1
Standard / soft law (1)
- United Kingdom (GB)
- Solely-automated AI decisions require contest, human review, explanation, and a DPIA (UK ICO) ICO UK GDPR guidance — Rights related to automated decision-making including profiling (Article 22)
Related incidents
No guardrail sits on this exact control; these incidents are cited by guardrails on related controls.
- UnitedHealth nH Predict claim-denial litigation (2023-11; alleged (not proven)). A class action filed in November 2023 alleges that UnitedHealth's nH Predict model had a 90% error rate, measured by denials reversed on appeal, while only about 0.2% of members appealed. UnitedHealth disputes the allegations; the litigation is ongoing. Source: STAT News · evidence grade: primary · cited by Monitor how often adverse AI decisions are reversed, and suspend models that are usually wrong
- Cigna PXDX batch claim denials (reported) (2022; alleged (not proven)). ProPublica, citing internal Cigna records, reported that Cigna's PXDX system was used to reject more than 300,000 claims over two months in 2022, with physicians spending an average of 1.2 seconds on each. Cigna disputes the reporting; related lawsuits are ongoing. Source: ProPublica / The Capitol Forum · evidence grade: press of record · cited by Make human review of adverse AI decisions substantive, not nominal