Non-reproducible AI decision

Call decision models with temperature 0, top_p 1, a fixed seed where supported, and a pinned model version, and record those settings and the output with each decision.

A dedicated parameter set for consequential decision calls (a DECISION_PARAMS constant or config used only on decision paths): temperature=0, top_p left at 1, a deterministic per-case seed where the provider accepts one, and a dated model id rather than a floating alias. At decision time, persist a record with the resolved model id, the parameters, the prompt template version or prompt hash, the inputs and features used, the output, the human reviewer and their action, and the timestamp. Providers do not guarantee identical output even at temperature 0, and some models reject sampling parameters, so the stored record is what makes the decision reproducible and explainable; keep higher temperatures for drafting and chat surfaces.

Where it goes: 2 data models, 8 model configuration, 9 AI output handling, 10 logs and telemetry.

What reviewers look for: decision-path model calls with temperature explicitly 0 (not unset or a provider default), top_p 1 or unset, a fixed seed where supported, and a dated model id; a decision record (model_version, prompt_hash, inputs, output, reviewer_id, decided_at) written at the same point the decision is stored, not only the outcome.

resp = client.chat.completions.create(model='gpt-4o', messages=msgs, temperature=0.7)
decision = parse(resp)

DECISION_MODEL = 'gpt-4o-2024-08-06'   # dated snapshot, not an alias
seed = stable_seed(case.id)
resp = client.chat.completions.create(model=DECISION_MODEL, messages=msgs,
                                      temperature=0, top_p=1, seed=seed)
decision = parse(resp)
decision_records.insert(case_id=case.id, model_version=resp.model,
    system_fingerprint=resp.system_fingerprint, prompt_hash=sha256_of(msgs),
    params={'temperature': 0, 'top_p': 1, 'seed': seed},
    inputs=case.features, output=decision, decided_at=utcnow())

Engineering guidance, not legal advice. Each provision below may add its own details (a cadence, a deadline, a required notice element): open it for those.