Make human review of adverse AI decisions substantive, not nominal

TwinEthos recommendation — not law

This is TwinEthos's opinion of what a responsible AI integration does anyway. It is never a legal or standards requirement; where binding law applies, the law governs.

The recommended-guardrail rule files are open under CC BY 4.0; attribution and scope are in the terms.

Informational data, not legal advice. Summaries are TwinEthos's own words and rules have not been reviewed by a lawyer: check the official text before relying on any of it. A guard addresses an item; adding it is not a statement that your code meets any law.

Evidence grade

Law in force in 1 jurisdiction, coming in 1 more

Law in force in 1 jurisdiction · law coming in 1 more · 1 standard or framework · 2 graded incidents.

TwinEthos recommendation, not law. Where binding law applies, the law governs. Binding law on this control, or in provisions cited as convergence, is in force in 1 jurisdiction (EU). Such law is enacted but not yet applicable, or stayed, in 1 more (US-CO). 1 standard or framework recommends it (IMDA Agentic AI MGF). 2 graded incidents cited.

Law in force on this control or cited as convergence

No solely-automated significant decision without human-intervention safeguards (GDPR Art. 22) (European Union (EU); GDPR Article 22(1); cited)

Law enacted, not yet applying

Consumers can request human review and data correction after an adverse ADMT decision (Colorado (US-CO); C.R.S. 6-1-1705(1); applies from 2027-01-01; stayed; cited)
High-risk AI systems must be designed for effective human oversight with override and stop (EU AI Act Art. 14) (European Union (EU); Article 14; applies from 2027-12-02; cited)

Standards and frameworks

Human involvement in AI decisions should be calibrated to harm; decisions should be explainable or repeatable (Singapore MGF) (Singapore (SG); Singapore Model AI Governance Framework (2nd ed.) — paras. 2.7(a), 3.13-3.15 (human involvement), 3.30 (repeatability); cited)

Family “AI decisions lack effective human review, override, or contest”: binding law on related controls is in force in Quebec (CA-QC), European Union (EU), Illinois (US-IL), Texas (US-TX); enacted, not yet applying in Colorado (US-CO). Context only: it does not change this guardrail's grade.

Graded incidents

UnitedHealth nH Predict claim-denial litigation (2023-11; alleged (not proven)) STAT News · evidence grade: primary
Cigna PXDX batch claim denials (reported) (2022; alleged (not proven)) ProPublica / The Capitol Forum · evidence grade: press of record

The guard to add

Replace bulk approval of AI outputs with per-item review that shows the evidence, enforces a minimum review time, and tracks override and agreement rates.

In the review workflow, each AI-proposed adverse decision is reviewed one at a time on a screen that shows the evidence the model used and lets the reviewer change the outcome; approve_all or bulk_approve endpoints for AI outputs are removed. The server records each review's reviewer, outcome (agree or override), and time spent, and rejects submissions faster than a minimum review time or beyond a per-reviewer throughput ceiling sized to the stakes. Dashboards track override_rate, agreement_rate, and review_duration per reviewer and queue, with alerts when agreement approaches 100% or review time approaches zero.

Example (FastAPI + prometheus_client), before:

@app.post('/reviews/approve_all')
def approve_all(queue_id: str):
    for item in queue.pending(queue_id):
        item.approve()

After:

MIN_REVIEW_SECONDS = 90   # sized to the stakes of this queue
REVIEWS = Counter('ai_reviews_total', 'Reviews of AI decisions', ['queue', 'result'])
REVIEW_TIME = Histogram('ai_review_duration_seconds', 'Time per review', ['queue'])

@app.post('/reviews/{item_id}')
def submit_review(item_id: str, body: ReviewIn, reviewer=Depends(current_reviewer)):
    item = queue.get(item_id)
    elapsed = time.time() - item.evidence_opened_at(reviewer.id)
    if elapsed < MIN_REVIEW_SECONDS:
        raise HTTPException(409, 'open the evidence and review before deciding')
    item.decide(reviewer.id, outcome=body.outcome, reason=body.reason)
    result = 'agree' if body.outcome == item.ai_outcome else 'override'
    REVIEWS.labels(queue=item.queue, result=result).inc()
    REVIEW_TIME.labels(queue=item.queue).observe(elapsed)

Control: Human review of AI decisions is nominal rather than substantive. Engineering guidance, not legal advice.

Why

Several laws require human review of automated decisions; none define what makes it real. Reported sign-off at about a second per claim is human review in name only. The control that matters is the substance of review, and it is measurable.

Class: law derived · set: human review substance · maturity: reviewed · confidence: high · id guardrail.review-substantive-human-review