Recommended guardrail
Scope every AI response and session cache to the requesting user or tenant
Key every cache of model responses, prompts, embeddings, or per-user AI state by the requesting user or tenant, and check ownership on every read. Detect caches keyed only by prompt or content hash that serve user-specific data, and session or history stores read without an owner check.
This is TwinEthos's opinion of what a responsible AI integration does anyway. It is never a legal or standards requirement; where binding law applies, the law governs.
The recommended-guardrail rule files are open under CC BY 4.0; attribution and scope are in the terms.
Informational data, not legal advice. Summaries are TwinEthos's own words and rules have not been reviewed by a lawyer: check the official text before relying on any of it. A guard addresses an item; adding it is not a statement that your code meets any law.
Evidence grade
Ahead of the law: no law yet, 2 incidents
2 graded incidents.
TwinEthos recommendation, not law. Where binding law applies, the law governs. No binding law in the corpus requires this control yet. 2 graded incidents cited.
Family “AI controls are not preserved under cost, latency, or model-change pressure”: binding law on related controls is in force in no jurisdiction. Context only: it does not change this guardrail's grade.
Graded incidents
- Prompt caches shared across all users at several LLM API providers (2024-09; confirmed) Gu et al., 'Auditing Prompt Caching in Language Model APIs' (original researchers) · evidence grade: primary
- ChatGPT cache bug exposed other users' chat titles and payment details (2023-03-20; disclosed by the operator) OpenAI (operator postmortem, 2023-03-24) · evidence grade: primary
The guard to add
Key every cache of model responses, embeddings, or session state by tenant and user, and check the stored owner against the requester on each read before serving it.
In the cache layer in front of the model call or the conversation store, build every key from the tenant id and user id plus the content digest, so identical prompts from different users never share an entry, and store the owner id with each entry. On read, compare the stored owner with the authenticated requester and treat a mismatch as a miss (or a 404 for history). Do not install a process-wide LLM cache (LangChain set_llm_cache, @lru_cache on a function whose output carries user data) for user-specific answers; a shared cache is acceptable only for content derived from non-personal inputs that every user receives alike.
Example (Python + redis-py), before:
key = 'resp:' + hashlib.sha256(prompt.encode()).hexdigest()
if (cached := r.get(key)) is not None:
return {'reply': cached.decode()}
reply = ask_model(user, prompt)
r.set(key, reply, ex=3600)After:
digest = hashlib.sha256(prompt.encode()).hexdigest()
key = f'resp:{user.tenant_id}:{user.id}:{digest}'
if (cached := r.get(key)) is not None:
return {'reply': cached.decode()}
reply = ask_model(user, prompt)
r.set(key, reply, ex=3600)Control: Cached AI responses or per-user AI state not scoped to the requester. Engineering guidance, not legal advice.
Why
A cache is the cheapest latency win in an AI stack and an easy way to hand one user's data to another. OpenAI has disclosed a cache fault that showed users other users' chat-history titles and payment details, and researchers have reported prompt caches shared across all users that could leak information between tenants.
Class: operational integrity · set: operational integrity · maturity: reviewed · confidence: high · id guardrail.opint-scoped-response-cache