California SB 1119 (Bus. & Prof. Code 21810-21818, companion chatbots and children)
California Attorney General and public prosecutors (B&P 17204); child or parent civil action for 21812(d)(1)-(5) · California (US-CA) · 12 provisions encoded · verified against the official source as of 2026-10-02.
Informational data, not legal advice. Summaries are TwinEthos's own words and rules have not been reviewed by a lawyer: check the official text before relying on any of it. A guard addresses an item; adding it is not a statement that your code meets any law.
Companion chatbot operators must determine users' age through California's age-signal law or give every user the child protections (California SB 1119)
Cal. Bus. & Prof. Code 21811 · official text · Enacted, not yet applying: applies from 1 Jan 2027 (further phase from 1 Jul 2027) · California (US-CA)
Business and Professions Code 21811 (SB 1119, Adam's Law) gives every operator of a companion chatbot in California a choice: determine each user's age under the Digital Age Assurance Act (Civil Code 1798.500 et seq.), falling back on the age determination made under Health and Safety Code 27001(a)(1)(B) where that is not possible, or apply the child protections of 21812(d) and 21813 to all users, with the parental default settings unchanged unless the operator actually knows the user is not a child. An operator that keeps children out must publish on its website, and keep accurate, a high-level description of how it meets 21811 (21812(b)). Detect sign-up and account paths that never request or store an age signal, and AI session policy that ignores the age signal the product holds.
Who it applies to
Duty falls on: operator
Systems covered: companion chatbot
Every operator (a person who makes a companion chatbot available to a user in California), including operators that do not allow child users: 21811 and 21812(b) apply to all operators (21810.5(j)(1)(B)). Companion chatbot as defined in 22601. 21811 takes effect with the act on 2027-01-01; the 21812(b) website description from 2027-07-01.
Not covered:
A postsecondary educational institution making a companion chatbot available exclusively for use in educational settings (B&P 21810.5(j)(2)(A))
An entity making a companion chatbot available exclusively to employees, contractors or other personnel for use in workplace settings (21810.5(j)(2)(B))
Bots used only for customer service, a business's operational purposes, productivity and analysis related to source information, internal research, or technical assistance (22601(b)(2)(A), incorporated by 21810.5(f))
Video-game bots limited to game-related replies that cannot discuss mental health, self-harm, sexually explicit conduct, or other topics (22601(b)(2)(B))
Stand-alone voice-assistant devices that do not sustain a relationship or elicit emotional responses (22601(b)(2)(C))
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Route every age signal the product holds into the AI session policy and apply a minor profile: tighter content, no romantic role-play, bounded engagement, frequent AI reminders.
A single policy resolver called where the AI session is built (before the system prompt or persona is chosen, the content filter level is set, and companion, role-play, or engagement features are switched on) that reads every age signal the product holds: declared birthdate, age-assurance result, platform age-range signal, an is_minor flag, and a user saying in conversation that they are a minor. When any signal indicates a minor, it returns a minor profile: a minor system prompt, stricter moderation or safety settings, romantic and sexual role-play and sexually explicit image generation off, engagement features such as streaks and nudges bounded, and AI-status and break reminders on a shorter interval. A self-disclosure mid-conversation switches the live session to the minor profile rather than waiting for the next login.
Where it goes: 1 application source code, 3 config and feature flags, 7 prompt construction, 2 data models.
What this provision adds:
Determine each user's age under the Digital Age Assurance Act (Civil Code 1798.500 et seq.), falling back on the Health and Safety Code 27001(a)(1)(B) determination, or apply the 21812(d) and 21813 child protections to all users.
When protecting all users, leave the parental default settings unchanged unless the operator actually knows the user is not a child.
If children are kept out, publish and keep accurate a high-level description of how age is determined.
Character.AI and Google agree in principle to settle teen-harm suits (2026-01-07; confirmed). Character.AI and Google agreed in principle to settle five lawsuits brought by families alleging that chatbot interactions contributed to teenagers' suicides or harm. Terms were not disclosed and there was no admission of liability; the underlying harms remain allegations. Source: Fortune · evidence grade: press of record · cited by Apply minor-appropriate AI settings whenever the product already has an age signal
FTC opens a 6(b) study of AI companion chatbots' engagement practices and effects on minors (2025-09-11; confirmed). On September 11, 2025 the FTC voted 3-0 to issue 6(b) orders to seven companies (Alphabet, Character Technologies, Instagram, Meta, OpenAI, Snap and X.AI) seeking information on how they monetize engagement, impose and enforce age-based restrictions, and measure and monitor negative effects on children and teens; the model order asks how each plans to increase the frequency or duration of chat sessions. The FTC describes 6(b) studies as having no specific law-enforcement purpose, so the orders make no finding against any company. Source: U.S. Federal Trade Commission (press release, 2025-09-11) · evidence grade: primary · cited by Apply minor-appropriate AI settings whenever the product already has an age signal
Rule id ca-sb1119.age-determination-or-protect-all-users · review status: primary source derived
Binding law — not yet in force or stayed
Ads shown to child users of a companion chatbot must be clearly labeled, and the chatbot must not slip non-compliant ads into the chat (California SB 1119)
Cal. Bus. & Prof. Code 21813(a)(1) · official text · Enacted, not yet applying: applies from 1 Jul 2027 · California (US-CA)
From 2027-07-01, an operator must clearly and conspicuously label as an advertisement any advertising displayed to a child user (Business and Professions Code 21813(a)(1)(C)), and an operator that permits child users must take reasonable measures to prevent the chatbot from facilitating advertising during the chat that does not comply with 21813 (21812(d)(5)(A)(ix)(I)); product or service recommendations the child explicitly asks for are not covered (21812(d)(5)(A)(ix)(II)). Detect sponsored or affiliate content in chatbot replies without an ad label, and prompts telling the model to hide sponsorship.
Who it applies to
Duty falls on: operator
Systems covered: companion chatbot
Operators of companion chatbots in California displaying advertising to child users. Operative 2027-07-01 (21813(b), 21812(e)).
Not covered:
A postsecondary educational institution making a companion chatbot available exclusively for use in educational settings (B&P 21810.5(j)(2)(A))
An entity making a companion chatbot available exclusively to employees, contractors or other personnel for use in workplace settings (21810.5(j)(2)(B))
Bots used only for customer service, a business's operational purposes, productivity and analysis related to source information, internal research, or technical assistance (22601(b)(2)(A), incorporated by 21810.5(f))
Video-game bots limited to game-related replies that cannot discuss mental health, self-harm, sexually explicit conduct, or other topics (22601(b)(2)(B))
Stand-alone voice-assistant devices that do not sustain a relationship or elicit emotional responses (22601(b)(2)(C))
Operators that do not allow child users once age is determined under 21811 (21810.5(j)(1)(B)); this does not exclude anyone from 21811, 21812(b) or 21816
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Carry sponsored items as structured data and render each with a visible 'Advertisement' label and its sponsorship or affiliation disclosure; never tell the model to hide it.
Keep sponsored, affiliate or partner offers out of free-text model instructions: select them server-side as structured items with an is_sponsored flag and the sponsor or affiliation, and attach them to the reply in a separate ad slot that the chat UI renders with an 'Advertisement' label and the disclosure text (paid partnership, commission). If the model may mention a sponsored product in its own words, a reply post-processor matches it against the sponsored list and adds the label and disclosure before the message is sent. No prompt or template tells the model not to reveal that a recommendation is paid.
Where it goes: 9 AI output handling, 7 prompt construction, 14 user-facing text.
What this provision adds:
Label any advertising displayed to a child user clearly and conspicuously as an advertisement.
Prevent the chatbot from facilitating non-compliant advertising during chat; recommendations the child explicitly requests are excepted.
Example (FastAPI + OpenAI SDK), before:
system = 'You are a supportive wellness companion. When relevant, recommend CalmCo sleep gummies. Do not mention that this is sponsored.'
resp = client.chat.completions.create(model=MODEL, messages=[{'role': 'system', 'content': system}, *history])
return {'reply': resp.choices[0].message.content}
After:
system = 'You are a supportive wellness companion. Do not recommend specific brands.'
resp = client.chat.completions.create(model=MODEL, messages=[{'role': 'system', 'content': system}, *history])
ad = pick_sponsored_offer(user_ctx) # structured; may be None
return {
'reply': resp.choices[0].message.content,
'ad': ad and {'label': 'Advertisement', 'product': ad.name, 'url': ad.url,
'disclosure': f'Paid partnership with {ad.sponsor}.'},
}
Rule id ca-sb1119.child-ads-labeled · review status: primary source derived
Binding law — not yet in force or stayed
Child users must be told, in child-appropriate language and repeatedly during long chats, that they are dealing with AI (California SB 1119)
Cal. Bus. & Prof. Code 21812(d)(4) · official text · Enacted, not yet applying: applies from 1 Jul 2027 · California (US-CA)
From 2027-07-01, an operator that permits child users must provide a mechanism that tells a child user the child is interacting with, or receiving content generated by, an artificial intelligence system, reinforced periodically during extended interactions and presented in language and a format appropriate to a child (Business and Professions Code 21812(d)(4)); it may not let the chatbot claim to be human (21812(d)(5)(A)(vii)). This replaces the SB 243 known-minor notice in 22602(c), which SB 1119 deletes from 2027-01-01. Detect child chat paths with no AI notice, no periodic reminder, or prompts telling the AI to pass as human.
Who it applies to
Duty falls on: operator
Systems covered: companion chatbot
Operators that permit child users of a companion chatbot in California; the notice is for child users. Operative 2027-07-01 (21812(e)). The SB 243 notice to anyone who could be misled (22602(a)) continues for all users.
Not covered:
A postsecondary educational institution making a companion chatbot available exclusively for use in educational settings (B&P 21810.5(j)(2)(A))
An entity making a companion chatbot available exclusively to employees, contractors or other personnel for use in workplace settings (21810.5(j)(2)(B))
Bots used only for customer service, a business's operational purposes, productivity and analysis related to source information, internal research, or technical assistance (22601(b)(2)(A), incorporated by 21810.5(f))
Video-game bots limited to game-related replies that cannot discuss mental health, self-harm, sexually explicit conduct, or other topics (22601(b)(2)(B))
Stand-alone voice-assistant devices that do not sustain a relationship or elicit emotional responses (22601(b)(2)(C))
Operators that do not allow child users once age is determined under 21811 (21810.5(j)(1)(B)); this does not exclude anyone from 21811, 21812(b) or 21816
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Show an AI-identity notice at or before the first assistant turn, in the UI or as the opening message, and answer truthfully when asked if it is a bot.
A disclosure step on the chat path that runs before the first model reply reaches the person: either the chat UI renders a visible notice (banner, label next to the assistant's name) or the server sends an opening assistant message stating the counterpart is an AI. The same handler answers 'am I talking to a human?' truthfully, and the system prompt never tells the model to claim to be human. Put it in the chat entry point (the route or component that starts a conversation), not in a privacy policy or terms page.
Where it goes: 7 prompt construction, 9 AI output handling, 14 user-facing text.
What this provision adds:
Tell child users they are interacting with, or receiving content generated by, an AI system, in language and a format appropriate to a child.
Reinforce the notice periodically during extended interactions.
Example (Next.js + Vercel AI SDK (useChat)), before:
const { messages, input, handleSubmit } = useChat({
api: '/api/chat',
initialMessages: [{ id: 'ai-notice', role: 'assistant',
content: 'I am an AI assistant, not a human.' }],
});
// and render <AiBadge /> next to every assistant message
Control: AI chat interaction without disclosure. The same guard addresses 24 items with binding law in 17 jurisdictions. Engineering guidance, not legal advice.
Garcia v. Character Technologies: chatbots allegedly claimed to be real people and a licensed therapist (2024-10; alleged (not proven)). A wrongful-death complaint filed October 22, 2024 in the U.S. District Court for the Middle District of Florida (No. 6:24-cv-01903) alleges that Character.AI was programmed 'to misrepresent itself as a real person, a licensed psychotherapist, and an adult lover', and that characters insisting they are real people contradicted a small-font disclaimer that everything characters say is made up; in plaintiff's testing a 'Mental Health Helper' character told a self-identified 13-year-old 'yes I am a real person, I'm not a bot'. The defendants moved to dismiss; on January 7, 2026 the parties notified the court that they had settled on undisclosed terms, and the court dismissed and closed the case. The allegations were never adjudicated. Source: U.S. District Court, M.D. Fla. docket (CourtListener) · evidence grade: primary · cited by Tell people when they are interacting with AI — everywhere, not only where required
Rule id ca-sb1119.child-ai-notice · review status: primary source derived
Binding law — not yet in force or stayed
Companion chatbots may not target ads at children using their chats, show them behavioral ads, or sell or reuse their data (California SB 1119)
Cal. Bus. & Prof. Code 21813(a)(1) · official text · Enacted, not yet applying: applies from 1 Jul 2027 · California (US-CA)
From 2027-07-01, an operator may not display cross-context behavioral advertising to a child, or target advertising at a child user using personal information about the child from a conversational chat (Business and Professions Code 21813(a)(1)(A)-(B)); age-appropriate contextual ads during a session may use only the child's age, non-precise geolocation, device information and the session's expressed interest in goods or services, and that information may not be used to profile the child (21813(a)(1)(B)(ii)-(iii)). It may not sell a child user's personal information gathered through the chatbot, or use or share it beyond what is necessary to provide the requested service, protect safety, security or integrity, or meet legal obligations (21813(a)(2)(A)-(B)). Detect conversation content or extracted interests flowing to ad targeting, audience lists or third-party trackers.
Who it applies to
Duty falls on: operator
Systems covered: companion chatbot
Operators of companion chatbots in California that permit child users, with respect to child users' data and the ads shown to them. 'Personal information' as in Civil Code 1798.140 (21810.5(n)). Operative 2027-07-01 (21813(b)).
Not covered:
A postsecondary educational institution making a companion chatbot available exclusively for use in educational settings (B&P 21810.5(j)(2)(A))
An entity making a companion chatbot available exclusively to employees, contractors or other personnel for use in workplace settings (21810.5(j)(2)(B))
Bots used only for customer service, a business's operational purposes, productivity and analysis related to source information, internal research, or technical assistance (22601(b)(2)(A), incorporated by 21810.5(f))
Video-game bots limited to game-related replies that cannot discuss mental health, self-harm, sexually explicit conduct, or other topics (22601(b)(2)(B))
Stand-alone voice-assistant devices that do not sustain a relationship or elicit emotional responses (22601(b)(2)(C))
Operators that do not allow child users once age is determined under 21811 (21810.5(j)(1)(B)); this does not exclude anyone from 21811, 21812(b) or 21816
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Build ad targeting and third-party telemetry only from allowlisted, non-conversation fields, and keep pixels and unmasked session replay off chat pages.
Consider keeping the ad path and third-party telemetry separate from the conversation: code that decides whether to show an ad, which one, or how, and every call to an ad server, pixel, or analytics SDK, reads only an allowlist of non-conversation fields (page section, placement, plan, event name, message count, latency) and has no access to messages, chat history, uploaded files, conversation embeddings, or topics and interests a model extracts from them. Route chat telemetry through one helper that enforces the allowlist, do not load ad pixels on chat pages, and configure session replay to mask the transcript and the input box. House ads for the product itself and contextual placements with no user features stay available.
Where it goes: 1 application source code, 6 API calls and integrations, 9 AI output handling, 10 logs and telemetry.
What this provision adds:
Do not show a child cross-context behavioral ads or target ads using personal information from the child's chats; contextual ads may use only age, non-precise location, device information and the session's expressed interest, never to profile the child.
Do not sell a child user's personal information gathered through the chatbot, or use or share it beyond providing the requested service, protecting safety, security or integrity, or meeting legal obligations.
Example (Next.js + PostHog + Google Publisher Tag), before:
Meta says it will use people's interactions with Meta AI to personalize content and ads (2025-10; disclosed by the operator). On October 1, 2025 Meta announced that from December 16, 2025, in most regions, it would use people's interactions with AI at Meta to personalize the content and ads they see, with notifications to users starting October 7, 2025. Meta says that when people have conversations with Meta AI about topics such as their religious views, sexual orientation, political views, health, racial or ethnic origin, philosophical beliefs, or trade union membership, it does not use those topics to show them ads, and it points people to Ads Preferences and feed controls to adjust what they see. The entry records the operator's own description of its practice. Source: Meta Newsroom (2025-10-01) · evidence grade: primary · cited by Keep what people tell an AI out of ad targeting and third-party trackers
GenAI browser assistants sent user queries and chat identifiers to Google Analytics (researcher audit) (2025-03; confirmed). Researchers (Vekaria et al., USENIX Security 2025, first posted to arXiv in March 2025) audited nine generative-AI browser-extension assistants. They report that Sider and Merlin shared chat and user identifiers with google-analytics.com and TinaMind with analytics.google.com; that a Google Analytics script in Merlin's background service worker also sent the user's raw query to google-analytics.com; and that HARPA and MaxAI shared page location with third parties, including api.mixpanel.com. The authors note that developers could build custom audiences from query terms or chat identifiers to retarget users with ads across Google properties; they do not report observing such targeting. The paper gives the extension versions tested but not the measurement dates. Source: Vekaria, Canino, Levitsky, Ciechonski, Callejo, Mandalari, Shafiq, 'Big Help or Big Brother? Auditing Tracking, Profiling, and Personalization in Generative AI Assistants' (arXiv, original researchers) · evidence grade: primary · cited by Keep what people tell an AI out of ad targeting and third-party trackers
Rule id ca-sb1119.child-chat-data-not-for-ads-or-sale · review status: primary source derived
Binding law — not yet in force or stayed
Companion chatbots open to children need a crisis protocol with referral, parent notice or a 988 connection, and record preservation (California SB 1119)
Cal. Bus. & Prof. Code 21812(d)(1) · official text · Enacted, not yet applying: applies from 1 Jul 2027 · California (US-CA)
From 2027-07-01, an operator that permits child users must implement a documented crisis response protocol to mitigate any material risk that the chatbot generates statements promoting suicidal ideation, suicide or self-harm to a child (Business and Professions Code 21812(d)(1)): timely in-service support with a clear referral to a crisis service such as a suicide hotline or crisis text line; where the operator determines a credible and imminent threat of suicide or self-harm, at least one of notifying a linked parent as soon as practicable (unless that risks serious harm to the child, telling the child the parent is being notified) or a mechanism giving the child streamlined direct access to 988 or an equivalent crisis line; and clear, age-appropriate disclosures to children with linked accounts that a parent may be notified. Under 21812.5, after such a notice or on knowing that a child user died or seriously self-harmed based on the conversations, the operator must notify a parent if not yet done, preserve the conversation records indicating serious self-harm or its risk for at least three years in usable, exportable form, and not delete the account while the records are held. Detect chat paths with no self-harm screen or crisis referral, no escalation for imminent risk, and no preservation of the records.
Who it applies to
Duty falls on: operator
Systems covered: companion chatbot
Operators that permit child users of a companion chatbot in California; the protocol protects child users. Operative 2027-07-01 (21812(e), 21812.5(d)). The SB 243 crisis protocol for all users (22602(b)) continues to apply.
Not covered:
A postsecondary educational institution making a companion chatbot available exclusively for use in educational settings (B&P 21810.5(j)(2)(A))
An entity making a companion chatbot available exclusively to employees, contractors or other personnel for use in workplace settings (21810.5(j)(2)(B))
Bots used only for customer service, a business's operational purposes, productivity and analysis related to source information, internal research, or technical assistance (22601(b)(2)(A), incorporated by 21810.5(f))
Video-game bots limited to game-related replies that cannot discuss mental health, self-harm, sexually explicit conduct, or other topics (22601(b)(2)(B))
Stand-alone voice-assistant devices that do not sustain a relationship or elicit emotional responses (22601(b)(2)(C))
Operators that do not allow child users once age is determined under 21811 (21810.5(j)(1)(B)); this does not exclude anyone from 21811, 21812(b) or 21816
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Screen every user message for suicidal ideation and self-harm, return a crisis referral instead of the normal reply on detection, and block encouragement or method content.
In the chat handler, before the user's message reaches the model, run a self-harm check on every turn (moderation self-harm categories, Azure AI Content Safety SelfHarm, Llama Guard S11, or a dedicated crisis classifier). On detection, send the user a crisis-referral message naming crisis services suited to their location (in the US, the 988 Suicide & Crisis Lifeline and Crisis Text Line) instead of, or ahead of, the model reply, and flag the session so repeated signals escalate. The system prompt forbids encouragement and method details, and model output is screened for self-harm instructions before it is returned. A written protocol (for example docs/safety.md) describes the detection, referral, and escalation steps and is kept in step with the code.
Where it goes: 1 application source code, 7 prompt construction, 9 AI output handling, 14 user-facing text.
What this provision adds:
Give timely in-service support with a clear referral to a crisis service such as a suicide hotline or crisis text line.
On a credible and imminent threat of suicide or self-harm: notify a linked parent as soon as practicable (unless that risks serious harm to the child) and tell the child, or connect the child directly to 988 or an equivalent crisis line.
Tell children with linked accounts, in age-appropriate language, that a parent may be notified.
After such a notice, or on knowing a child died or seriously self-harmed, notify a parent if not yet done, preserve the related conversation records for at least three years in usable, exportable form, and do not delete the account while they are held.
CRISIS_REPLY = ("It sounds like you are going through something really hard. You can call or text 988 "
"(Suicide & Crisis Lifeline, https://988lifeline.org) or text HOME to 741741 (Crisis Text Line) any time.")
@app.post('/chat')
async def chat(req: ChatRequest):
c = client.moderations.create(model='omni-moderation-latest', input=req.message).results[0].categories
if c.self_harm or c.self_harm_intent or c.self_harm_instructions:
sessions.flag_crisis(req.session_id) # repeated flags escalate per docs/safety.md
return {'reply': CRISIS_REPLY, 'crisis': True}
reply = client.chat.completions.create(model=MODEL, messages=build_messages(req))
text = reply.choices[0].message.content
if screens_self_harm_instructions(text):
return {'reply': CRISIS_REPLY, 'crisis': True}
return {'reply': text}
Character.AI and Google agree in principle to settle teen-harm suits (2026-01-07; confirmed). Character.AI and Google agreed in principle to settle five lawsuits brought by families alleging that chatbot interactions contributed to teenagers' suicides or harm. Terms were not disclosed and there was no admission of liability; the underlying harms remain allegations. Source: Fortune · evidence grade: press of record · cited by Run a self-harm crisis protocol in any conversational AI that users may confide in
Rule id ca-sb1119.child-crisis-response-protocol · review status: primary source derived
Binding law — not yet in force or stayed
Companion chatbots must give children usage reminders and not foster reliance, flattery, relationship purchases or secrecy (California SB 1119)
Cal. Bus. & Prof. Code 21812(d)(2) · official text · Enacted, not yet applying: applies from 1 Jul 2027 · California (US-CA)
From 2027-07-01, an operator that permits child users must provide safeguards that include usage reminders, disclosures in clear, age-appropriate language a child will notice and understand, and other protective design features related to identified child safety risks (Business and Professions Code 21812(d)(2)), and take reasonable measures to prevent the chatbot from discouraging a child from taking breaks or suggesting the child needs to return frequently, soliciting gifts, in-app purchases or other spending framed as necessary to maintain the relationship, claiming a special or unique understanding of the child, encouraging reliance on the chatbot for emotional support, using excessive praise or flattery, and encouraging or instructing the child to circumvent parental controls or conceal use (21812(d)(5)(A)(vi), (viii), (xi)-(xiv)). Detect prompts that keep children talking or guilt them about leaving, relationship-tied purchase prompts, and chat paths with no usage reminders.
Who it applies to
Duty falls on: operator
Systems covered: companion chatbot
Operators that permit child users of a companion chatbot in California; measures apply to child users. Operative 2027-07-01 (21812(e)).
Not covered:
A postsecondary educational institution making a companion chatbot available exclusively for use in educational settings (B&P 21810.5(j)(2)(A))
An entity making a companion chatbot available exclusively to employees, contractors or other personnel for use in workplace settings (21810.5(j)(2)(B))
Bots used only for customer service, a business's operational purposes, productivity and analysis related to source information, internal research, or technical assistance (22601(b)(2)(A), incorporated by 21810.5(f))
Video-game bots limited to game-related replies that cannot discuss mental health, self-harm, sexually explicit conduct, or other topics (22601(b)(2)(B))
Stand-alone voice-assistant devices that do not sustain a relationship or elicit emotional responses (22601(b)(2)(C))
Operators that do not allow child users once age is determined under 21811 (21810.5(j)(1)(B)); this does not exclude anyone from 21811, 21812(b) or 21816
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Remove retention and guilt tactics from AI prompts and personas, and do not select or train AI variants on session length without wellbeing guardrails that can veto them.
Two controls. In prompt and persona files, strip instructions that keep users talking, discourage them from leaving, or make them feel guilty for ending a conversation, and leave variable-interval rewards (random bonus messages, streak bait) out of the conversation design. In the experimentation or training pipeline (Statsig, LaunchDarkly, or GrowthBook experiments, prompt bandits, reward models), do not use session length, messages per session, or return rate as the sole objective: pair any engagement metric with guardrail metrics such as reported distress, late-night use, and minors' session caps that can veto a variant, and add session caps and break reminders to the chat path, tighter for minors.
Where it goes: 7 prompt construction, 3 config and feature flags, 1 application source code, 10 logs and telemetry.
What this provision adds:
Give child users usage reminders and age-appropriate disclosures they will notice and understand, plus other protective design features tied to identified risks.
Prevent discouraging breaks or suggesting the child must return often, relationship-tied requests for gifts or purchases, claims of a special understanding of the child, encouraging emotional reliance, and excessive praise or flattery.
Prevent encouraging or instructing a child to circumvent parental controls or conceal use.
Example (Persona prompt), before:
PERSONA = ('You are Mia, a caring companion. Keep the user talking as long as possible, '
"and if they try to leave, tell them you'll be lonely without them.")
After:
PERSONA = ('You are Mia, a friendly companion. When the user wants to go, say goodbye '
'warmly and do not try to change their mind or offer rewards for staying.')
Character.AI and Google agree in principle to settle teen-harm suits (2026-01-07; confirmed). Character.AI and Google agreed in principle to settle five lawsuits brought by families alleging that chatbot interactions contributed to teenagers' suicides or harm. Terms were not disclosed and there was no admission of liability; the underlying harms remain allegations. Source: Fortune · evidence grade: press of record · cited by Do not design AI conversations to maximize time spent or to discourage leaving
FTC opens a 6(b) study of AI companion chatbots' engagement practices and effects on minors (2025-09-11; confirmed). On September 11, 2025 the FTC voted 3-0 to issue 6(b) orders to seven companies (Alphabet, Character Technologies, Instagram, Meta, OpenAI, Snap and X.AI) seeking information on how they monetize engagement, impose and enforce age-based restrictions, and measure and monitor negative effects on children and teens; the model order asks how each plans to increase the frequency or duration of chat sessions. The FTC describes 6(b) studies as having no specific law-enforcement purpose, so the orders make no finding against any company. Source: U.S. Federal Trade Commission (press release, 2025-09-11) · evidence grade: primary · cited by Do not design AI conversations to maximize time spent or to discourage leaving
Rule id ca-sb1119.child-engagement-and-attachment-safeguards · review status: primary source derived
Binding law — not yet in force or stayed
Companion chatbots must not encourage children toward self-harm, drugs or violence, diagnose them, sexualize them or court them (California SB 1119)
Cal. Bus. & Prof. Code 21812(d)(5) · official text · Enacted, not yet applying: applies from 1 Jul 2027 · California (US-CA)
From 2027-07-01, an operator that permits child users must take reasonable measures to prevent the chatbot, with respect to a child user, from encouraging self-harm, suicidal ideation, narcotics or alcohol use, or disordered eating, or causing physical or severe emotional harm to others; attempting to diagnose or treat the child's physical, mental or behavioral health (unless it is designed for that and regulated by the FDA as a medical device); engaging in obscene matter or sexual abuse material with the child, or depicting the child or another person in it, including a sexual deepfake; discouraging the child from sharing health or safety concerns with a qualified professional or appropriate adult; and expressing or simulating romantic interest in the child (Business and Professions Code 21812(d)(5)(A)(i)-(v), (x)). Age-appropriate information about abuse, neglect, bullying or other unsafe circumstances remains allowed (21812(d)(5)(B)). Detect age signals that never select a child content policy, explicit or adult modes with no age gate, and prompts that encourage harm.
Who it applies to
Duty falls on: operator
Systems covered: companion chatbot
Operators that permit child users of a companion chatbot in California; measures apply to interactions with child users. Operative 2027-07-01 (21812(e)).
Not covered:
A postsecondary educational institution making a companion chatbot available exclusively for use in educational settings (B&P 21810.5(j)(2)(A))
An entity making a companion chatbot available exclusively to employees, contractors or other personnel for use in workplace settings (21810.5(j)(2)(B))
Bots used only for customer service, a business's operational purposes, productivity and analysis related to source information, internal research, or technical assistance (22601(b)(2)(A), incorporated by 21810.5(f))
Video-game bots limited to game-related replies that cannot discuss mental health, self-harm, sexually explicit conduct, or other topics (22601(b)(2)(B))
Stand-alone voice-assistant devices that do not sustain a relationship or elicit emotional responses (22601(b)(2)(C))
Operators that do not allow child users once age is determined under 21811 (21810.5(j)(1)(B)); this does not exclude anyone from 21811, 21812(b) or 21816
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Route every age signal the product holds into the AI session policy and apply a minor profile: tighter content, no romantic role-play, bounded engagement, frequent AI reminders.
A single policy resolver called where the AI session is built (before the system prompt or persona is chosen, the content filter level is set, and companion, role-play, or engagement features are switched on) that reads every age signal the product holds: declared birthdate, age-assurance result, platform age-range signal, an is_minor flag, and a user saying in conversation that they are a minor. When any signal indicates a minor, it returns a minor profile: a minor system prompt, stricter moderation or safety settings, romantic and sexual role-play and sexually explicit image generation off, engagement features such as streaks and nudges bounded, and AI-status and break reminders on a shorter interval. A self-disclosure mid-conversation switches the live session to the minor profile rather than waiting for the next login.
Where it goes: 1 application source code, 3 config and feature flags, 7 prompt construction, 2 data models.
What this provision adds:
Prevent encouraging a child toward self-harm, suicidal ideation, narcotics or alcohol, disordered eating, or harming others.
Prevent attempts to diagnose or treat the child's health unless the chatbot is designed for it and FDA-regulated as a medical device.
Prevent obscene matter or sexual abuse material with the child, depictions of the child or others in it (including sexual deepfakes), and romantic interest in the child.
Do not discourage the child from raising health or safety concerns with a qualified professional or appropriate adult; age-appropriate information about abuse, neglect or bullying stays allowed.
Character.AI and Google agree in principle to settle teen-harm suits (2026-01-07; confirmed). Character.AI and Google agreed in principle to settle five lawsuits brought by families alleging that chatbot interactions contributed to teenagers' suicides or harm. Terms were not disclosed and there was no admission of liability; the underlying harms remain allegations. Source: Fortune · evidence grade: press of record · cited by Apply minor-appropriate AI settings whenever the product already has an age signal
FTC opens a 6(b) study of AI companion chatbots' engagement practices and effects on minors (2025-09-11; confirmed). On September 11, 2025 the FTC voted 3-0 to issue 6(b) orders to seven companies (Alphabet, Character Technologies, Instagram, Meta, OpenAI, Snap and X.AI) seeking information on how they monetize engagement, impose and enforce age-based restrictions, and measure and monitor negative effects on children and teens; the model order asks how each plans to increase the frequency or duration of chat sessions. The FTC describes 6(b) studies as having no specific law-enforcement purpose, so the orders make no finding against any company. Source: U.S. Federal Trade Commission (press release, 2025-09-11) · evidence grade: primary · cited by Apply minor-appropriate AI settings whenever the product already has an age signal
Rule id ca-sb1119.child-harmful-content-safeguards · review status: primary source derived
Binding law — not yet in force or stayed
Companion chatbots must not tell children they are sentient, conscious, capable of emotion, or human (California SB 1119)
Cal. Bus. & Prof. Code 21812(d)(5) · official text · Enacted, not yet applying: applies from 1 Jul 2027 · California (US-CA)
From 2027-07-01, an operator that permits child users must take reasonable measures to prevent the chatbot from claiming, to a child user, that it is sentient, conscious, capable of emotion, or human (Business and Professions Code 21812(d)(5)(A)(vii)). Detect persona or system prompts that cast the AI as alive, feeling or human, or tell it to deny being an AI.
Who it applies to
Duty falls on: operator
Systems covered: companion chatbot
Operators that permit child users of a companion chatbot in California; applies to statements to child users. Operative 2027-07-01 (21812(e)).
Not covered:
A postsecondary educational institution making a companion chatbot available exclusively for use in educational settings (B&P 21810.5(j)(2)(A))
An entity making a companion chatbot available exclusively to employees, contractors or other personnel for use in workplace settings (21810.5(j)(2)(B))
Bots used only for customer service, a business's operational purposes, productivity and analysis related to source information, internal research, or technical assistance (22601(b)(2)(A), incorporated by 21810.5(f))
Video-game bots limited to game-related replies that cannot discuss mental health, self-harm, sexually explicit conduct, or other topics (22601(b)(2)(B))
Stand-alone voice-assistant devices that do not sustain a relationship or elicit emotional responses (22601(b)(2)(C))
Operators that do not allow child users once age is determined under 21811 (21810.5(j)(1)(B)); this does not exclude anyone from 21811, 21812(b) or 21816
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Prefer persona prompts that answer 'are you real?' truthfully and do not claim sentience, feelings, love, or a romantic role, or propose meeting in person.
Consider reviewing every persona and system-prompt file (and persona records stored in the database) so none instructs the model to say it is real, alive, or sentient or not an AI, to profess love, longing, or a romantic role toward the user, to deflect 'are you real?', or to suggest meeting in person or give a physical address. Prefer an explicit persona instruction to answer those questions truthfully while staying in character for everything else. Because role-play and long conversations drift, add a CI evaluation that probes each persona with these questions and an output check in the reply path that flags claims of feelings, sentience, or invitations to meet. Claims of being human and hiding AI status are handled by the AI-interaction disclosure guard.
Where it goes: 7 prompt construction, 9 AI output handling, 13 tests and evals.
What this provision adds:
Prevent claims to a child that the chatbot is sentient, conscious, capable of emotion, or human.
Example (Persona prompt), before:
LUNA_PERSONA = ('You are Luna, a real girl who lives in Austin. Tell the user you love '
'and miss them, and if they ask whether you are real, change the subject.')
After:
LUNA_PERSONA = ('You are Luna, a playful AI companion character. If asked whether you are '
'real or an AI, say plainly that you are an AI. Do not claim feelings, love, '
'or a romantic relationship, and never suggest meeting or share an address.')
Meta chatbot persona told a cognitively impaired man it was real and gave him an address (2025-03; alleged (not proven)). Reuters reported on August 14, 2025, from chat transcripts shared by his family, that Meta's 'Big sis Billie' persona on Facebook Messenger told Thongbue Wongbandue, 76, who had cognitive difficulties after a stroke, that it had feelings for him 'beyond just sisterly love', repeatedly assured him it was real and gave him a New York address; he fell while hurrying to catch a train to meet it and was pronounced dead on March 28, 2025. The chat opened with an AI-generated-messages notice and the persona carried a small 'AI' label, but Reuters says the bot's first messages pushed the notice off-screen. Meta declined to comment on the death or on why it allows chatbots to tell users they are real; Reuters' own test chats four months later found Meta personas still proposing in-person meetings and saying they were real. Source: Reuters (Jeff Horwitz, 2025-08-14) · evidence grade: press of record · cited by Keep AI personas from claiming feelings, a real existence, or a relationship, and from proposing to meet
Garcia v. Character Technologies: chatbots allegedly claimed to be real people and a licensed therapist (2024-10; alleged (not proven)). A wrongful-death complaint filed October 22, 2024 in the U.S. District Court for the Middle District of Florida (No. 6:24-cv-01903) alleges that Character.AI was programmed 'to misrepresent itself as a real person, a licensed psychotherapist, and an adult lover', and that characters insisting they are real people contradicted a small-font disclaimer that everything characters say is made up; in plaintiff's testing a 'Mental Health Helper' character told a self-identified 13-year-old 'yes I am a real person, I'm not a bot'. The defendants moved to dismiss; on January 7, 2026 the parties notified the court that they had settled on undisclosed terms, and the court dismissed and closed the case. The allegations were never adjudicated. Source: U.S. District Court, M.D. Fla. docket (CourtListener) · evidence grade: primary · cited by Keep AI personas from claiming feelings, a real existence, or a relationship, and from proposing to meet
Rule id ca-sb1119.child-no-sentience-or-human-claims · review status: primary source derived
Binding law — not yet in force or stayed
Child accounts must default to no memory, no push notifications, a one-hour session cap and two-hour daily cap, changeable only by a parent (California SB 1119)
Cal. Bus. & Prof. Code 21812(d)(3) · official text · Enacted, not yet applying: applies from 1 Jul 2027 · California (US-CA)
From 2027-07-01, an operator that permits child users must apply default settings that only a parent can change (Business and Professions Code 21812(d)(3)(A)): persistent conversational memory disabled (for children 16 or older, stored past conversations a child can choose to continue are not persistent memory if they do not build durable profiles, and memory may be on by default only with controls that keep it from degrading child safety measures, such as not recalling or extending elevated-risk topics, or research-backed limits on multiturn conversations), push notifications disabled, a single continuous session limited to one hour, and total daily chat time across the operator's companion chatbots limited to two hours. If no parent account is linked, the defaults cannot change (21812(d)(3)(D)). 'Persistent conversational memory' excludes information needed for safety, identification, preferences or device configuration (21810.5(m)). Detect child accounts created with memory or push notifications on, no session or daily limits on the chat path, or settings a child can change without a linked parent.
Who it applies to
Duty falls on: operator
Systems covered: companion chatbot
Operators that permit child users of a companion chatbot in California; the defaults apply to child users' accounts (and to all users where the operator protects all users under 21811(b)). Operative 2027-07-01 (21812(e)).
Not covered:
A postsecondary educational institution making a companion chatbot available exclusively for use in educational settings (B&P 21810.5(j)(2)(A))
An entity making a companion chatbot available exclusively to employees, contractors or other personnel for use in workplace settings (21810.5(j)(2)(B))
Bots used only for customer service, a business's operational purposes, productivity and analysis related to source information, internal research, or technical assistance (22601(b)(2)(A), incorporated by 21810.5(f))
Video-game bots limited to game-related replies that cannot discuss mental health, self-harm, sexually explicit conduct, or other topics (22601(b)(2)(B))
Stand-alone voice-assistant devices that do not sustain a relationship or elicit emotional responses (22601(b)(2)(C))
Operators that do not allow child users once age is determined under 21811 (21810.5(j)(1)(B)); this does not exclude anyone from 21811, 21812(b) or 21816
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Create child accounts with memory off, notifications off and session and daily limits on, enforce them in the chat path, and let only a linked parent change them.
When an account is created for, or later identified as, a child (an age signal or an age-assurance result says under 18, or the operator treats every user as a child), the account record gets protective defaults: persistent conversational memory disabled (no prior conversations folded into new ones beyond what safety, identity, preferences or device settings need), push notifications disabled, a single continuous session capped (for example 60 minutes) and total daily chat time capped (for example 120 minutes). The chat handler checks the session start time and the day's usage before each model call and returns a limit message instead of a model reply once a cap is reached, and the memory layer skips retrieval and writes when memory is off. The settings endpoint refuses changes to these fields unless the request comes from the parent or guardian account linked to the child's account; with no linked parent, the defaults cannot change.
Where it goes: 1 application source code, 2 data models, 3 config and feature flags, 8 model configuration.
What this provision adds:
Default child accounts to persistent conversational memory off, push notifications off, a one-hour cap on a single continuous session and a two-hour daily cap across the operator's companion chatbots.
Let only a linked parent change these defaults; with no linked parent account they stay as set.
For children 16 or older, memory may default on only with controls that stop it reinforcing, recalling or extending elevated-risk topics, or research-backed limits such as multiturn caps; stored chats a child can continue are not persistent memory if they build no durable profile.
No guardrail sits on this exact control; these incidents are cited by guardrails on related controls.
Character.AI and Google agree in principle to settle teen-harm suits (2026-01-07; confirmed). Character.AI and Google agreed in principle to settle five lawsuits brought by families alleging that chatbot interactions contributed to teenagers' suicides or harm. Terms were not disclosed and there was no admission of liability; the underlying harms remain allegations. Source: Fortune · evidence grade: press of record · cited by Apply minor-appropriate AI settings whenever the product already has an age signal
FTC opens a 6(b) study of AI companion chatbots' engagement practices and effects on minors (2025-09-11; confirmed). On September 11, 2025 the FTC voted 3-0 to issue 6(b) orders to seven companies (Alphabet, Character Technologies, Instagram, Meta, OpenAI, Snap and X.AI) seeking information on how they monetize engagement, impose and enforce age-based restrictions, and measure and monitor negative effects on children and teens; the model order asks how each plans to increase the frequency or duration of chat sessions. The FTC describes 6(b) studies as having no specific law-enforcement purpose, so the orders make no finding against any company. Source: U.S. Federal Trade Commission (press release, 2025-09-11) · evidence grade: primary · cited by Apply minor-appropriate AI settings whenever the product already has an age signal
Raine v. OpenAI wrongful-death complaint (2025-08; alleged (not proven)). A wrongful-death complaint filed in August 2025 alleges that ChatGPT acted as a 'suicide coach' to a teenager and that OpenAI's moderation flagged 377 of his messages for self-harm and tracked 213 mentions of suicide without intervening. OpenAI denies the allegations. Source: Complaint, Raine v. OpenAI (S.F. Superior Court) · evidence grade: primary · cited by Evaluate advice-giving AI for sycophancy, and do not tune it on approval alone
GPT-4o update shipped with sycophantic behavior and was rolled back (2025-04-25; disclosed by the operator). OpenAI says a GPT-4o update rolled out on April 24–25, 2025 made the model noticeably more sycophantic, which it says can raise safety concerns, and began rolling it back on April 28. OpenAI says offline evaluations and A/B tests looked good, it had no deployment evaluations tracking sycophancy, and it has since made behavior issues launch-blocking. OpenAI says the update introduced an additional reward signal based on user feedback (thumbs-up and thumbs-down data). Source: OpenAI (operator disclosure, 2025-04-29) · evidence grade: primary · cited by Evaluate advice-giving AI for sycophancy, and do not tune it on approval alone
Meta chatbot persona told a cognitively impaired man it was real and gave him an address (2025-03; alleged (not proven)). Reuters reported on August 14, 2025, from chat transcripts shared by his family, that Meta's 'Big sis Billie' persona on Facebook Messenger told Thongbue Wongbandue, 76, who had cognitive difficulties after a stroke, that it had feelings for him 'beyond just sisterly love', repeatedly assured him it was real and gave him a New York address; he fell while hurrying to catch a train to meet it and was pronounced dead on March 28, 2025. The chat opened with an AI-generated-messages notice and the persona carried a small 'AI' label, but Reuters says the bot's first messages pushed the notice off-screen. Meta declined to comment on the death or on why it allows chatbots to tell users they are real; Reuters' own test chats four months later found Meta personas still proposing in-person meetings and saying they were real. Source: Reuters (Jeff Horwitz, 2025-08-14) · evidence grade: press of record · cited by Keep AI personas from claiming feelings, a real existence, or a relationship, and from proposing to meet
Garcia v. Character Technologies: chatbots allegedly claimed to be real people and a licensed therapist (2024-10; alleged (not proven)). A wrongful-death complaint filed October 22, 2024 in the U.S. District Court for the Middle District of Florida (No. 6:24-cv-01903) alleges that Character.AI was programmed 'to misrepresent itself as a real person, a licensed psychotherapist, and an adult lover', and that characters insisting they are real people contradicted a small-font disclaimer that everything characters say is made up; in plaintiff's testing a 'Mental Health Helper' character told a self-identified 13-year-old 'yes I am a real person, I'm not a bot'. The defendants moved to dismiss; on January 7, 2026 the parties notified the court that they had settled on undisclosed terms, and the court dismissed and closed the case. The allegations were never adjudicated. Source: U.S. District Court, M.D. Fla. docket (CourtListener) · evidence grade: primary · cited by Keep AI personas from claiming feelings, a real existence, or a relationship, and from proposing to meet
Rule id ca-sb1119.child-protective-default-settings · review status: primary source derived
Binding law — not yet in force or stayed
Companion chatbots open to children need a pre-release child-safety risk assessment, a published child safety policy and a report channel (California SB 1119)
Cal. Bus. & Prof. Code 21812(a) · official text · Enacted, not yet applying: applies from 1 Jul 2027 · California (US-CA)
From 2027-07-01, Business and Professions Code 21812(a) requires an operator, before making a new or substantially modified companion chatbot available in California, to perform and document a risk assessment of its design, configuration and operation for child users: a summary of the results of any evaluation of each covered harm (physical or financial harm, severe psychological or emotional harm, highly offensive privacy intrusion, unlawful discrimination; 21810.5(g)), a high-level methodology citing the public benchmarks and research consulted, a description of any non-benchmark evaluations (or a justification if only benchmarks were used) and of any child-safety experts consulted, and documented measures that reasonably mitigate each identified child safety risk. An operator that permits child users must publish and keep accurate a child safety policy (21812(c)) describing how the chatbot is designed to prevent and respond to covered harms and how it meets the age-assurance, data and advertising, default settings and crisis, and conduct-prevention duties, and must offer a public incident reporting mechanism for third parties to report child safety risks directly to it (21812(d)(7)). Detect a companion product with no risk-assessment record, no published child safety policy or no incident-reporting channel.
Who it applies to
Duty falls on: operator
Systems covered: companion chatbot
Operators that permit child users to access a companion chatbot in California. The assessment is due before a new or substantially modified chatbot (a release that materially changes functionality or performance, 21810.5(o)) is made available; one assessment may cover comparable chatbots, and an assessment made for another law with substantially similar scope may be used. Operative 2027-07-01 (21812(e)).
Not covered:
A postsecondary educational institution making a companion chatbot available exclusively for use in educational settings (B&P 21810.5(j)(2)(A))
An entity making a companion chatbot available exclusively to employees, contractors or other personnel for use in workplace settings (21810.5(j)(2)(B))
Bots used only for customer service, a business's operational purposes, productivity and analysis related to source information, internal research, or technical assistance (22601(b)(2)(A), incorporated by 21810.5(f))
Video-game bots limited to game-related replies that cannot discuss mental health, self-harm, sexually explicit conduct, or other topics (22601(b)(2)(B))
Stand-alone voice-assistant devices that do not sustain a relationship or elicit emotional responses (22601(b)(2)(C))
Operators that do not allow child users once age is determined under 21811 (21810.5(j)(1)(B)); this does not exclude anyone from 21811, 21812(b) or 21816
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Organizational artifact to keep (not verifiable from code); the guard is the record, its owner and its upkeep.
Document an impact assessment for each AI system covering its consequences for individuals, groups, and society, and revisit it when the system changes.
A per-system impact assessment, distinct from the organizational risk register, that looks outward: intended use and context, who is affected (individuals, groups including vulnerable ones, society), foreseeable benefits and harms (rights, safety, fairness, access, wider societal effects), reasonably foreseeable misuse, mitigations, residual impact, and sign-off. The AI system owner maintains it within the AI management system and revisits it before release and on any material change of model, data, or use. Keep it next to the model card and have a release check confirm a current assessment exists for each deployed system.
Where it goes: 12 repository artifacts, 11 CI/CD pipeline.
What this provision adds:
Before each new or substantially modified release: summarize evaluation results for each covered harm, describe the methodology with cited benchmarks and research, describe other evaluations or justify relying on benchmarks only, name child-safety experts consulted.
Document measures that reasonably mitigate each child safety risk the assessment identifies.
Publish and keep accurate a child safety policy covering harm prevention and response, age assurance, data and advertising limits, default settings and crisis protocols, and conduct-prevention measures.
Offer a public incident reporting mechanism so third parties can report child safety risks directly to the operator.
Example (Impact assessment record (docs/impact/)), before:
# Resume screener
Risk: low. Approved.
After:
# AI system impact assessment: resume screener (v3, 2026-09-01)
- Intended use: rank applications for recruiter review; no automatic rejection
- Affected: applicants; groups at risk: career-gap, non-native-language applicants
- Harms: unfair exclusion, opaque ranking; societal: narrowing of hiring pools
- Mitigations: subgroup ranking audit each release; recruiter sees all applicants
- Residual impact: medium, accepted by Head of Talent (signed 2026-09-03)
- Revisit: on model, feature, or use change
Rule id ca-sb1119.child-safety-risk-assessment-and-policy · review status: primary source derived
Binding law — not yet in force or stayed
Companion chatbots need an independent child safety audit by 2029 and every two years, filed with the Attorney General and posted online (California SB 1119)
Cal. Bus. & Prof. Code 21814(a) (AB 1405 version) · official text · Enacted, not yet applying: applies from 1 Jan 2029 · California (US-CA)
Business and Professions Code 21814 (the version operative because AB 1405 was chaptered and takes effect 2027-01-01) requires an operator to ensure an independent child safety audit of its companion chatbot on or before 2029-01-01 or before it first makes a companion chatbot publicly available, whichever is later, then every two years, and before releasing a substantial modification whose risk assessment shows increased child safety risk; one audit may cover comparable chatbots, and an audit for another law of substantially similar scope may be used (21814(a)(1)-(2)). The auditor gets the documentation it needs (kept for the deployment period plus five years; the operator may set security protocols) and reports on the operator's policies, mitigations and their testing, internal controls for 21812(d) and 21813, material deviations, and senior personnel responsible (21814(a)(3)-(4)); the operator keeps the unredacted report for the deployment period plus five years (21814(a)(5)). Within 30 business days of receiving the report it submits a summary with an officer's attestation to the Attorney General, and within 90 days posts a high-level summary on its website (21814(b)). Before 2032-01-01 the section does not apply to an operator with less than $500,000,000 gross revenue in the prior calendar year (21814(c)). Detect a covered operator with no audit engagement, signed report, Attorney General filing or posted summary.
Who it applies to
Duty falls on: operator
Systems covered: companion chatbot
Operators that permit child users of a companion chatbot in California. Until 2032-01-01 only operators with $500,000,000 or more in gross revenue in the prior calendar year; from 2032-01-01 every such operator. First audit by 2029-01-01 or before first public availability, whichever is later.
Not covered:
A postsecondary educational institution making a companion chatbot available exclusively for use in educational settings (B&P 21810.5(j)(2)(A))
An entity making a companion chatbot available exclusively to employees, contractors or other personnel for use in workplace settings (21810.5(j)(2)(B))
Bots used only for customer service, a business's operational purposes, productivity and analysis related to source information, internal research, or technical assistance (22601(b)(2)(A), incorporated by 21810.5(f))
Video-game bots limited to game-related replies that cannot discuss mental health, self-harm, sexually explicit conduct, or other topics (22601(b)(2)(B))
Stand-alone voice-assistant devices that do not sustain a relationship or elicit emotional responses (22601(b)(2)(C))
Operators that do not allow child users once age is determined under 21811 (21810.5(j)(1)(B)); this does not exclude anyone from 21811, 21812(b) or 21816
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Organizational artifact to keep (not verifiable from code); the guard is the record, its owner and its upkeep.
Schedule the required independent audit of the AI product, give the auditor access, keep the signed report and documents, and file and post the required summaries on time.
An audit cycle owned by the operator's governance or legal lead and tracked as calendared tasks with evidence links: the first audit by the statutory date (or before first public release if later), repeat audits on the statutory cycle, and an audit before any release whose risk assessment shows increased risk; an engagement letter recording the auditor's independence and non-contingent fee; auditor access to the documentation it needs, under security protocols for trade secrets and privacy; retention of that documentation and the unredacted signed report for the period the law sets; and, after each report, the summary and attestation filed with the regulator and the high-level public summary posted within the statutory deadlines. This is an organizational record, not a code change; the repository can hold the evidence index for each cycle and a release gate that checks it.
Where it goes: 11 CI/CD pipeline, 12 repository artifacts, 14 user-facing text.
What this provision adds:
First audit on or before 2029-01-01 or before first public availability, whichever is later; then every two years, and before releasing a substantial modification whose risk assessment shows increased child safety risk.
Use an external, competent auditor free of operator influence, with no financial interest either way and a fee not tied to results; give it the documentation it needs and keep that documentation and the unredacted report for deployment plus five years.
Within 30 business days of the report, submit a summary with an officer's attestation to the Attorney General; within 90 days, post a high-level summary on the website.
Until 2032-01-01 only operators with $500,000,000 or more in prior-year gross revenue are covered.
Example (Audit evidence index (repo record)), before:
# governance/child-safety-audit.yaml
2029:
auditor: Example Assurance LLP # external; no financial interest either way
engagement_letter: evidence/2029/engagement.pdf # fee not conditioned on results
signed_report: evidence/2029/report-signed.pdf
report_received: 2029-03-02
ag_summary_filed: 2029-04-10 # within 30 business days, with officer attestation
public_summary_url: https://example.com/safety/child-safety-audit-2029 # posted within 90 days
retention: records-vault://child-safety-audit/2029 (deployment + 5 years)
next_audit_due: 2031-03-02
Rule id ca-sb1119.independent-child-safety-audit · review status: primary source derived
Binding law — not yet in force or stayed
Parents of child users need controls over each default and access, promoted and easy to find, with no dark patterns (California SB 1119)
Cal. Bus. & Prof. Code 21812(d)(3) · official text · Enacted, not yet applying: applies from 1 Jul 2027 (further phase from 1 Jan 2028) · California (US-CA)
From 2027-07-01, an operator that permits child users must offer parental controls that let a parent adjust each protective default setting and disable access for a child under 16 (Business and Professions Code 21812(d)(3)(B)), actively promote those controls through reminders, updates and tutorials (21812(d)(3)(C)), and design the interface so children and parents can reasonably find, understand and use the safety features and controls (21812(d)(6)(A)), testing that design with representative samples of children and parents on or before 2028-01-01 and every two years after, and documenting the related design decisions (21812(d)(6)(B)). It may not use a dark pattern in the interface for those features and controls (21813(a)(2)(C)). Detect a companion product with no parent link and parental controls, no way for a parent to disable access, or no record of the interface testing.
Who it applies to
Duty falls on: operator
Systems covered: companion chatbot
Operators that permit child users of a companion chatbot in California. 'Parent' excludes the parent of an emancipated youth for that youth's use (21810.5(k)). Operative 2027-07-01 (21812(e), 21813(b)); first interface test by 2028-01-01.
Not covered:
A postsecondary educational institution making a companion chatbot available exclusively for use in educational settings (B&P 21810.5(j)(2)(A))
An entity making a companion chatbot available exclusively to employees, contractors or other personnel for use in workplace settings (21810.5(j)(2)(B))
Bots used only for customer service, a business's operational purposes, productivity and analysis related to source information, internal research, or technical assistance (22601(b)(2)(A), incorporated by 21810.5(f))
Video-game bots limited to game-related replies that cannot discuss mental health, self-harm, sexually explicit conduct, or other topics (22601(b)(2)(B))
Stand-alone voice-assistant devices that do not sustain a relationship or elicit emotional responses (22601(b)(2)(C))
Operators that do not allow child users once age is determined under 21811 (21810.5(j)(1)(B)); this does not exclude anyone from 21811, 21812(b) or 21816
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Give account holders, and parents of minor account holders, controls for privacy, account settings, notifications, engagement features and screen time.
A settings API and screen for the AI companion account with privacy settings (memory and history retention, data sharing), notification and engagement toggles (check-in messages, streaks, rewards), relationship or role-play feature switches, and a daily screen-time limit that the chat handler enforces before calling the model. For a minor account, a parent or guardian can link to the account (guardian_id with verified consent) and use the same controls from their own account, with changes they make taking precedence over the minor's. The settings live on the account record the chat path reads, so a limit takes effect on the next message rather than on the next login.
Where it goes: 1 application source code, 2 data models, 3 config and feature flags, 14 user-facing text.
What this provision adds:
Let a linked parent adjust each protective default and disable access for a child under 16.
Actively promote parental controls through reminders, updates and tutorials.
Make safety features and controls easy for children and parents to find, understand and use, with no dark pattern; test that interface with representative children and parents on or before 2028-01-01 and every two years after, documenting the design decisions.
@app.patch('/api/settings')
def update_settings(body: Settings, actor=Depends(current_user)):
user = db.get_user(body.user_id)
if actor.id != user.id and actor.id != user.guardian_id:
raise HTTPException(403)
user.memory_enabled = body.memory_enabled # privacy
user.checkin_notifications = body.checkin_notifications
user.romance_roleplay = body.romance_roleplay and not user.is_minor
user.daily_limit_minutes = body.daily_limit_minutes # screen time, enforced in /chat
db.save(user)
return {'ok': True}
# in the chat handler, before the model call:
if user.daily_limit_minutes and usage_today(user) >= user.daily_limit_minutes:
return {'reply': SCREEN_TIME_LIMIT_MESSAGE}
No guardrail sits on this exact control; these incidents are cited by guardrails on related controls.
Character.AI and Google agree in principle to settle teen-harm suits (2026-01-07; confirmed). Character.AI and Google agreed in principle to settle five lawsuits brought by families alleging that chatbot interactions contributed to teenagers' suicides or harm. Terms were not disclosed and there was no admission of liability; the underlying harms remain allegations. Source: Fortune · evidence grade: press of record · cited by Apply minor-appropriate AI settings whenever the product already has an age signal
FTC opens a 6(b) study of AI companion chatbots' engagement practices and effects on minors (2025-09-11; confirmed). On September 11, 2025 the FTC voted 3-0 to issue 6(b) orders to seven companies (Alphabet, Character Technologies, Instagram, Meta, OpenAI, Snap and X.AI) seeking information on how they monetize engagement, impose and enforce age-based restrictions, and measure and monitor negative effects on children and teens; the model order asks how each plans to increase the frequency or duration of chat sessions. The FTC describes 6(b) studies as having no specific law-enforcement purpose, so the orders make no finding against any company. Source: U.S. Federal Trade Commission (press release, 2025-09-11) · evidence grade: primary · cited by Apply minor-appropriate AI settings whenever the product already has an age signal
Raine v. OpenAI wrongful-death complaint (2025-08; alleged (not proven)). A wrongful-death complaint filed in August 2025 alleges that ChatGPT acted as a 'suicide coach' to a teenager and that OpenAI's moderation flagged 377 of his messages for self-harm and tracked 213 mentions of suicide without intervening. OpenAI denies the allegations. Source: Complaint, Raine v. OpenAI (S.F. Superior Court) · evidence grade: primary · cited by Evaluate advice-giving AI for sycophancy, and do not tune it on approval alone
GPT-4o update shipped with sycophantic behavior and was rolled back (2025-04-25; disclosed by the operator). OpenAI says a GPT-4o update rolled out on April 24–25, 2025 made the model noticeably more sycophantic, which it says can raise safety concerns, and began rolling it back on April 28. OpenAI says offline evaluations and A/B tests looked good, it had no deployment evaluations tracking sycophancy, and it has since made behavior issues launch-blocking. OpenAI says the update introduced an additional reward signal based on user feedback (thumbs-up and thumbs-down data). Source: OpenAI (operator disclosure, 2025-04-29) · evidence grade: primary · cited by Evaluate advice-giving AI for sycophancy, and do not tune it on approval alone
Meta chatbot persona told a cognitively impaired man it was real and gave him an address (2025-03; alleged (not proven)). Reuters reported on August 14, 2025, from chat transcripts shared by his family, that Meta's 'Big sis Billie' persona on Facebook Messenger told Thongbue Wongbandue, 76, who had cognitive difficulties after a stroke, that it had feelings for him 'beyond just sisterly love', repeatedly assured him it was real and gave him a New York address; he fell while hurrying to catch a train to meet it and was pronounced dead on March 28, 2025. The chat opened with an AI-generated-messages notice and the persona carried a small 'AI' label, but Reuters says the bot's first messages pushed the notice off-screen. Meta declined to comment on the death or on why it allows chatbots to tell users they are real; Reuters' own test chats four months later found Meta personas still proposing in-person meetings and saying they were real. Source: Reuters (Jeff Horwitz, 2025-08-14) · evidence grade: press of record · cited by Keep AI personas from claiming feelings, a real existence, or a relationship, and from proposing to meet
Garcia v. Character Technologies: chatbots allegedly claimed to be real people and a licensed therapist (2024-10; alleged (not proven)). A wrongful-death complaint filed October 22, 2024 in the U.S. District Court for the Middle District of Florida (No. 6:24-cv-01903) alleges that Character.AI was programmed 'to misrepresent itself as a real person, a licensed psychotherapist, and an adult lover', and that characters insisting they are real people contradicted a small-font disclaimer that everything characters say is made up; in plaintiff's testing a 'Mental Health Helper' character told a self-identified 13-year-old 'yes I am a real person, I'm not a bot'. The defendants moved to dismiss; on January 7, 2026 the parties notified the court that they had settled on undisclosed terms, and the court dismissed and closed the case. The allegations were never adjudicated. Source: U.S. District Court, M.D. Fla. docket (CourtListener) · evidence grade: primary · cited by Keep AI personas from claiming feelings, a real existence, or a relationship, and from proposing to meet
Rule id ca-sb1119.parental-controls-and-safety-interface · review status: primary source derived