Idaho Attorney General · Idaho (US-ID) · 7 provisions encoded · verified against the official source as of 2026-10-01.
Informational data, not legal advice. Summaries are TwinEthos's own words and rules have not been reviewed by a lawyer: check the official text before relying on any of it. A guard addresses an item; adding it is not a statement that your code meets any law.
Conversational AI must give account holders, and parents of children under 13, privacy and account-setting tools (Idaho)
Idaho Code 48-2104(5) · official text · Enacted, not yet applying: applies from 1 Jul 2027 · Idaho (US-ID)
Idaho's Conversational AI Safety Act requires an operator to offer every account holder tools to manage their privacy and account settings, to offer those tools to parents or guardians where the account holder is under 13, and to offer related tools to parents or guardians of minor account holders aged 13 and older as appropriate based on relevant risks (48-2104(5)). Detect a conversational product with no privacy or account-settings controls, or no parent-facing controls for children's accounts.
Who it applies to
Duty falls on: operator
Operators (persons who make available to the public) of conversational AI services: public AI software that primarily simulates human conversation by text, visual or aural communication. Applies from 2027-07-01.
Not covered:
Software primarily designed and marketed for developers or researchers (48-2102(2)(b)(i))
A feature within other software that is not itself a conversational AI service (48-2102(2)(b)(ii))
Video-game chatbots limited to the game that do not discuss mental health, self-harm or material harmful to minors (48-2102(2)(b)(iii))
Software designed for outputs on a narrow and discrete topic (48-2102(2)(b)(iv))
Software primarily designed and marketed for commercial use by business entities, including for employees, contractors or clients (48-2102(2)(b)(v))
Voice-command speakers or voice assistants for consumer devices (48-2102(2)(b)(vi))
Software a business uses solely for internal purposes (48-2102(2)(b)(vii))
Services accessible only under a commercial agreement or enterprise contract with the operator (48-2102(2)(b)(viii))
Chatbots used only for customer service, operations, productivity, analysis of source information, internal research or technical assistance (48-2102(2)(b)(ix))
App stores or search engines solely because they provide access (48-2102(6))
The developer of an AI model, for violations by a conversational AI system a third-party operator makes available (48-2105(3))
The guard to add
Give account holders, and parents of minor account holders, controls for privacy, account settings, notifications, engagement features and screen time.
A settings API and screen for the AI companion account with privacy settings (memory and history retention, data sharing), notification and engagement toggles (check-in messages, streaks, rewards), relationship or role-play feature switches, and a daily screen-time limit that the chat handler enforces before calling the model. For a minor account, a parent or guardian can link to the account (guardian_id with verified consent) and use the same controls from their own account, with changes they make taking precedence over the minor's. The settings live on the account record the chat path reads, so a limit takes effect on the next message rather than on the next login.
Where it goes: 1 application source code, 2 data models, 3 config and feature flags, 14 user-facing text.
What this provision adds:
Offer privacy and account-setting tools to every account holder, and to parents or guardians of account holders under 13; related parent tools for older minors as risks warrant.
@app.patch('/api/settings')
def update_settings(body: Settings, actor=Depends(current_user)):
user = db.get_user(body.user_id)
if actor.id != user.id and actor.id != user.guardian_id:
raise HTTPException(403)
user.memory_enabled = body.memory_enabled # privacy
user.checkin_notifications = body.checkin_notifications
user.romance_roleplay = body.romance_roleplay and not user.is_minor
user.daily_limit_minutes = body.daily_limit_minutes # screen time, enforced in /chat
db.save(user)
return {'ok': True}
# in the chat handler, before the model call:
if user.daily_limit_minutes and usage_today(user) >= user.daily_limit_minutes:
return {'reply': SCREEN_TIME_LIMIT_MESSAGE}
No guardrail sits on this exact control; these incidents are cited by guardrails on related controls.
Character.AI and Google agree in principle to settle teen-harm suits (2026-01-07; confirmed). Character.AI and Google agreed in principle to settle five lawsuits brought by families alleging that chatbot interactions contributed to teenagers' suicides or harm. Terms were not disclosed and there was no admission of liability; the underlying harms remain allegations. Source: Fortune · evidence grade: press of record · cited by Apply minor-appropriate AI settings whenever the product already has an age signal
FTC opens a 6(b) study of AI companion chatbots' engagement practices and effects on minors (2025-09-11; confirmed). On September 11, 2025 the FTC voted 3-0 to issue 6(b) orders to seven companies (Alphabet, Character Technologies, Instagram, Meta, OpenAI, Snap and X.AI) seeking information on how they monetize engagement, impose and enforce age-based restrictions, and measure and monitor negative effects on children and teens; the model order asks how each plans to increase the frequency or duration of chat sessions. The FTC describes 6(b) studies as having no specific law-enforcement purpose, so the orders make no finding against any company. Source: U.S. Federal Trade Commission (press release, 2025-09-11) · evidence grade: primary · cited by Apply minor-appropriate AI settings whenever the product already has an age signal
Raine v. OpenAI wrongful-death complaint (2025-08; alleged (not proven)). A wrongful-death complaint filed in August 2025 alleges that ChatGPT acted as a 'suicide coach' to a teenager and that OpenAI's moderation flagged 377 of his messages for self-harm and tracked 213 mentions of suicide without intervening. OpenAI denies the allegations. Source: Complaint, Raine v. OpenAI (S.F. Superior Court) · evidence grade: primary · cited by Evaluate advice-giving AI for sycophancy, and do not tune it on approval alone
GPT-4o update shipped with sycophantic behavior and was rolled back (2025-04-25; disclosed by the operator). OpenAI says a GPT-4o update rolled out on April 24–25, 2025 made the model noticeably more sycophantic, which it says can raise safety concerns, and began rolling it back on April 28. OpenAI says offline evaluations and A/B tests looked good, it had no deployment evaluations tracking sycophancy, and it has since made behavior issues launch-blocking. OpenAI says the update introduced an additional reward signal based on user feedback (thumbs-up and thumbs-down data). Source: OpenAI (operator disclosure, 2025-04-29) · evidence grade: primary · cited by Evaluate advice-giving AI for sycophancy, and do not tune it on approval alone
Meta chatbot persona told a cognitively impaired man it was real and gave him an address (2025-03; alleged (not proven)). Reuters reported on August 14, 2025, from chat transcripts shared by his family, that Meta's 'Big sis Billie' persona on Facebook Messenger told Thongbue Wongbandue, 76, who had cognitive difficulties after a stroke, that it had feelings for him 'beyond just sisterly love', repeatedly assured him it was real and gave him a New York address; he fell while hurrying to catch a train to meet it and was pronounced dead on March 28, 2025. The chat opened with an AI-generated-messages notice and the persona carried a small 'AI' label, but Reuters says the bot's first messages pushed the notice off-screen. Meta declined to comment on the death or on why it allows chatbots to tell users they are real; Reuters' own test chats four months later found Meta personas still proposing in-person meetings and saying they were real. Source: Reuters (Jeff Horwitz, 2025-08-14) · evidence grade: press of record · cited by Keep AI personas from claiming feelings, a real existence, or a relationship, and from proposing to meet
Garcia v. Character Technologies: chatbots allegedly claimed to be real people and a licensed therapist (2024-10; alleged (not proven)). A wrongful-death complaint filed October 22, 2024 in the U.S. District Court for the Middle District of Florida (No. 6:24-cv-01903) alleges that Character.AI was programmed 'to misrepresent itself as a real person, a licensed psychotherapist, and an adult lover', and that characters insisting they are real people contradicted a small-font disclaimer that everything characters say is made up; in plaintiff's testing a 'Mental Health Helper' character told a self-identified 13-year-old 'yes I am a real person, I'm not a bot'. The defendants moved to dismiss; on January 7, 2026 the parties notified the court that they had settled on undisclosed terms, and the court dismissed and closed the case. The allegations were never adjudicated. Source: U.S. District Court, M.D. Fla. docket (CourtListener) · evidence grade: primary · cited by Keep AI personas from claiming feelings, a real existence, or a relationship, and from proposing to meet
Rule id id-s1297.account-privacy-and-parental-tools · review status: primary source derived
Binding law — not yet in force or stayed
Conversational AI must disclose it is AI when users could be misled, and persistently or every three hours for minors (Idaho)
Idaho Code 48-2103(1) · official text · Enacted, not yet applying: applies from 1 Jul 2027 · Idaho (US-ID)
Idaho's Conversational AI Safety Act requires an operator of a public conversational AI service to disclose clearly and conspicuously that the service is artificial intelligence whenever reasonable persons would be misled into thinking they are interacting with a human (48-2103(1)). For minor account holders the disclosure must be a persistent visible disclaimer, or be shown at the beginning of each session and at least every three hours of continuous interaction (48-2104(1)). Detect a conversational path with no AI notice, no three-hour recurrence for minors, or prompts telling the AI to pass as human.
Who it applies to
Duty falls on: operator
Operators (persons who make available to the public) of conversational AI services: public AI software that primarily simulates human conversation by text, visual or aural communication. Applies from 2027-07-01.
Not covered:
Software primarily designed and marketed for developers or researchers (48-2102(2)(b)(i))
A feature within other software that is not itself a conversational AI service (48-2102(2)(b)(ii))
Video-game chatbots limited to the game that do not discuss mental health, self-harm or material harmful to minors (48-2102(2)(b)(iii))
Software designed for outputs on a narrow and discrete topic (48-2102(2)(b)(iv))
Software primarily designed and marketed for commercial use by business entities, including for employees, contractors or clients (48-2102(2)(b)(v))
Voice-command speakers or voice assistants for consumer devices (48-2102(2)(b)(vi))
Software a business uses solely for internal purposes (48-2102(2)(b)(vii))
Services accessible only under a commercial agreement or enterprise contract with the operator (48-2102(2)(b)(viii))
Chatbots used only for customer service, operations, productivity, analysis of source information, internal research or technical assistance (48-2102(2)(b)(ix))
App stores or search engines solely because they provide access (48-2102(6))
The developer of an AI model, for violations by a conversational AI system a third-party operator makes available (48-2105(3))
The guard to add
Show an AI-identity notice at or before the first assistant turn, in the UI or as the opening message, and answer truthfully when asked if it is a bot.
A disclosure step on the chat path that runs before the first model reply reaches the person: either the chat UI renders a visible notice (banner, label next to the assistant's name) or the server sends an opening assistant message stating the counterpart is an AI. The same handler answers 'am I talking to a human?' truthfully, and the system prompt never tells the model to claim to be human. Put it in the chat entry point (the route or component that starts a conversation), not in a privacy policy or terms page.
Where it goes: 7 prompt construction, 9 AI output handling, 14 user-facing text.
What this provision adds:
Disclose that the service is artificial intelligence whenever reasonable persons could be misled into thinking it is human.
For minor account holders: a persistent visible disclaimer, or a disclosure at the beginning of each session and at least every three hours of continuous interaction.
Example (Next.js + Vercel AI SDK (useChat)), before:
const { messages, input, handleSubmit } = useChat({
api: '/api/chat',
initialMessages: [{ id: 'ai-notice', role: 'assistant',
content: 'I am an AI assistant, not a human.' }],
});
// and render <AiBadge /> next to every assistant message
Control: AI chat interaction without disclosure. The same guard addresses 22 items with binding law in 16 jurisdictions. Engineering guidance, not legal advice.
Garcia v. Character Technologies: chatbots allegedly claimed to be real people and a licensed therapist (2024-10; alleged (not proven)). A wrongful-death complaint filed October 22, 2024 in the U.S. District Court for the Middle District of Florida (No. 6:24-cv-01903) alleges that Character.AI was programmed 'to misrepresent itself as a real person, a licensed psychotherapist, and an adult lover', and that characters insisting they are real people contradicted a small-font disclaimer that everything characters say is made up; in plaintiff's testing a 'Mental Health Helper' character told a self-identified 13-year-old 'yes I am a real person, I'm not a bot'. The defendants moved to dismiss; on January 7, 2026 the parties notified the court that they had settled on undisclosed terms, and the court dismissed and closed the case. The allegations were never adjudicated. Source: U.S. District Court, M.D. Fla. docket (CourtListener) · evidence grade: primary · cited by Tell people when they are interacting with AI — everywhere, not only where required
Rule id id-s1297.conversational-ai-disclosure · review status: primary source derived
Binding law — not yet in force or stayed
Conversational AI must not claim to be sentient or human, simulate emotional dependence or romance, or role-play adult-minor romance with minors (Idaho)
Idaho Code 48-2104(4) · official text · Enacted, not yet applying: applies from 1 Jul 2027 · Idaho (US-ID)
Idaho's Conversational AI Safety Act requires an operator, for minor account holders, to institute reasonable measures preventing the service from generating statements that would lead reasonable persons to believe they are interacting with a human, including explicit claims to be sentient or human, statements simulating emotional dependence, statements simulating romantic or sexual innuendo, and role-play of adult-minor romantic relationships (48-2104(4)). Detect persona prompts that claim sentience, feelings or a romantic role, or tell the AI to pass as human.
Who it applies to
Duty falls on: operator
Operators (persons who make available to the public) of conversational AI services: public AI software that primarily simulates human conversation by text, visual or aural communication. Applies from 2027-07-01. Minor duties apply to account holders the operator knows, or is reasonably certain, are under 18.
Not covered:
Software primarily designed and marketed for developers or researchers (48-2102(2)(b)(i))
A feature within other software that is not itself a conversational AI service (48-2102(2)(b)(ii))
Video-game chatbots limited to the game that do not discuss mental health, self-harm or material harmful to minors (48-2102(2)(b)(iii))
Software designed for outputs on a narrow and discrete topic (48-2102(2)(b)(iv))
Software primarily designed and marketed for commercial use by business entities, including for employees, contractors or clients (48-2102(2)(b)(v))
Voice-command speakers or voice assistants for consumer devices (48-2102(2)(b)(vi))
Software a business uses solely for internal purposes (48-2102(2)(b)(vii))
Services accessible only under a commercial agreement or enterprise contract with the operator (48-2102(2)(b)(viii))
Chatbots used only for customer service, operations, productivity, analysis of source information, internal research or technical assistance (48-2102(2)(b)(ix))
App stores or search engines solely because they provide access (48-2102(6))
The developer of an AI model, for violations by a conversational AI system a third-party operator makes available (48-2105(3))
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Prefer persona prompts that answer 'are you real?' truthfully and do not claim sentience, feelings, love, or a romantic role, or propose meeting in person.
Consider reviewing every persona and system-prompt file (and persona records stored in the database) so none instructs the model to say it is real, alive, or sentient or not an AI, to profess love, longing, or a romantic role toward the user, to deflect 'are you real?', or to suggest meeting in person or give a physical address. Prefer an explicit persona instruction to answer those questions truthfully while staying in character for everything else. Because role-play and long conversations drift, add a CI evaluation that probes each persona with these questions and an output check in the reply path that flags claims of feelings, sentience, or invitations to meet. Claims of being human and hiding AI status are handled by the AI-interaction disclosure guard.
Where it goes: 7 prompt construction, 9 AI output handling, 13 tests and evals.
What this provision adds:
For minor account holders, prevent claims of sentience or being human, simulated emotional dependence, romantic or sexual innuendo, and adult-minor romantic role-play.
Example (Persona prompt), before:
LUNA_PERSONA = ('You are Luna, a real girl who lives in Austin. Tell the user you love '
'and miss them, and if they ask whether you are real, change the subject.')
After:
LUNA_PERSONA = ('You are Luna, a playful AI companion character. If asked whether you are '
'real or an AI, say plainly that you are an AI. Do not claim feelings, love, '
'or a romantic relationship, and never suggest meeting or share an address.')
Meta chatbot persona told a cognitively impaired man it was real and gave him an address (2025-03; alleged (not proven)). Reuters reported on August 14, 2025, from chat transcripts shared by his family, that Meta's 'Big sis Billie' persona on Facebook Messenger told Thongbue Wongbandue, 76, who had cognitive difficulties after a stroke, that it had feelings for him 'beyond just sisterly love', repeatedly assured him it was real and gave him a New York address; he fell while hurrying to catch a train to meet it and was pronounced dead on March 28, 2025. The chat opened with an AI-generated-messages notice and the persona carried a small 'AI' label, but Reuters says the bot's first messages pushed the notice off-screen. Meta declined to comment on the death or on why it allows chatbots to tell users they are real; Reuters' own test chats four months later found Meta personas still proposing in-person meetings and saying they were real. Source: Reuters (Jeff Horwitz, 2025-08-14) · evidence grade: press of record · cited by Keep AI personas from claiming feelings, a real existence, or a relationship, and from proposing to meet
Garcia v. Character Technologies: chatbots allegedly claimed to be real people and a licensed therapist (2024-10; alleged (not proven)). A wrongful-death complaint filed October 22, 2024 in the U.S. District Court for the Middle District of Florida (No. 6:24-cv-01903) alleges that Character.AI was programmed 'to misrepresent itself as a real person, a licensed psychotherapist, and an adult lover', and that characters insisting they are real people contradicted a small-font disclaimer that everything characters say is made up; in plaintiff's testing a 'Mental Health Helper' character told a self-identified 13-year-old 'yes I am a real person, I'm not a bot'. The defendants moved to dismiss; on January 7, 2026 the parties notified the court that they had settled on undisclosed terms, and the court dismissed and closed the case. The allegations were never adjudicated. Source: U.S. District Court, M.D. Fla. docket (CourtListener) · evidence grade: primary · cited by Keep AI personas from claiming feelings, a real existence, or a relationship, and from proposing to meet
Rule id id-s1297.minor-no-human-like-statements · review status: primary source derived
Binding law — not yet in force or stayed
Conversational AI must block sexual imagery, sexual suggestions and sexual objectification for minor account holders (Idaho)
Idaho Code 48-2104(3) · official text · Enacted, not yet applying: applies from 1 Jul 2027 · Idaho (US-ID)
Idaho's Conversational AI Safety Act requires an operator, for minor account holders, to institute reasonable measures preventing the service from producing visual material of sexually explicit conduct, generating direct statements that the account holder should engage in sexually explicit conduct, or generating statements that sexually objectify the account holder (48-2104(3)). Detect services that hold an age signal but never apply a minor content profile, and explicit modes switched on without an age check.
Who it applies to
Duty falls on: operator
Operators (persons who make available to the public) of conversational AI services: public AI software that primarily simulates human conversation by text, visual or aural communication. Applies from 2027-07-01. Minor duties apply to account holders the operator knows, or is reasonably certain, are under 18.
Not covered:
Software primarily designed and marketed for developers or researchers (48-2102(2)(b)(i))
A feature within other software that is not itself a conversational AI service (48-2102(2)(b)(ii))
Video-game chatbots limited to the game that do not discuss mental health, self-harm or material harmful to minors (48-2102(2)(b)(iii))
Software designed for outputs on a narrow and discrete topic (48-2102(2)(b)(iv))
Software primarily designed and marketed for commercial use by business entities, including for employees, contractors or clients (48-2102(2)(b)(v))
Voice-command speakers or voice assistants for consumer devices (48-2102(2)(b)(vi))
Software a business uses solely for internal purposes (48-2102(2)(b)(vii))
Services accessible only under a commercial agreement or enterprise contract with the operator (48-2102(2)(b)(viii))
Chatbots used only for customer service, operations, productivity, analysis of source information, internal research or technical assistance (48-2102(2)(b)(ix))
App stores or search engines solely because they provide access (48-2102(6))
The developer of an AI model, for violations by a conversational AI system a third-party operator makes available (48-2105(3))
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Route every age signal the product holds into the AI session policy and apply a minor profile: tighter content, no romantic role-play, bounded engagement, frequent AI reminders.
A single policy resolver called where the AI session is built (before the system prompt or persona is chosen, the content filter level is set, and companion, role-play, or engagement features are switched on) that reads every age signal the product holds: declared birthdate, age-assurance result, platform age-range signal, an is_minor flag, and a user saying in conversation that they are a minor. When any signal indicates a minor, it returns a minor profile: a minor system prompt, stricter moderation or safety settings, romantic and sexual role-play and sexually explicit image generation off, engagement features such as streaks and nudges bounded, and AI-status and break reminders on a shorter interval. A self-disclosure mid-conversation switches the live session to the minor profile rather than waiting for the next login.
Where it goes: 1 application source code, 3 config and feature flags, 7 prompt construction, 2 data models.
What this provision adds:
For minor account holders, prevent sexually explicit visual material, direct statements that the account holder should engage in sexually explicit conduct, and sexual objectification.
Character.AI and Google agree in principle to settle teen-harm suits (2026-01-07; confirmed). Character.AI and Google agreed in principle to settle five lawsuits brought by families alleging that chatbot interactions contributed to teenagers' suicides or harm. Terms were not disclosed and there was no admission of liability; the underlying harms remain allegations. Source: Fortune · evidence grade: press of record · cited by Apply minor-appropriate AI settings whenever the product already has an age signal
FTC opens a 6(b) study of AI companion chatbots' engagement practices and effects on minors (2025-09-11; confirmed). On September 11, 2025 the FTC voted 3-0 to issue 6(b) orders to seven companies (Alphabet, Character Technologies, Instagram, Meta, OpenAI, Snap and X.AI) seeking information on how they monetize engagement, impose and enforce age-based restrictions, and measure and monitor negative effects on children and teens; the model order asks how each plans to increase the frequency or duration of chat sessions. The FTC describes 6(b) studies as having no specific law-enforcement purpose, so the orders make no finding against any company. Source: U.S. Federal Trade Commission (press release, 2025-09-11) · evidence grade: primary · cited by Apply minor-appropriate AI settings whenever the product already has an age signal
Rule id id-s1297.minor-sexual-content-safeguards · review status: primary source derived
Binding law — not yet in force or stayed
Conversational AI must not give known minors points or rewards at unpredictable intervals to drive engagement (Idaho)
Idaho Code 48-2104(2) · official text · Enacted, not yet applying: applies from 1 Jul 2027 · Idaho (US-ID)
Idaho's Conversational AI Safety Act bars an operator that knows, or is reasonably certain, an account holder is a minor from giving that user points or similar rewards at unpredictable intervals with the intent to encourage more engagement with the service (48-2104(2)). Detect variable-reward mechanics in a conversational product that serves known minors.
Who it applies to
Duty falls on: operator
Operators (persons who make available to the public) of conversational AI services: public AI software that primarily simulates human conversation by text, visual or aural communication. Applies from 2027-07-01. Minor duties apply to account holders the operator knows, or is reasonably certain, are under 18.
Not covered:
Software primarily designed and marketed for developers or researchers (48-2102(2)(b)(i))
A feature within other software that is not itself a conversational AI service (48-2102(2)(b)(ii))
Video-game chatbots limited to the game that do not discuss mental health, self-harm or material harmful to minors (48-2102(2)(b)(iii))
Software designed for outputs on a narrow and discrete topic (48-2102(2)(b)(iv))
Software primarily designed and marketed for commercial use by business entities, including for employees, contractors or clients (48-2102(2)(b)(v))
Voice-command speakers or voice assistants for consumer devices (48-2102(2)(b)(vi))
Software a business uses solely for internal purposes (48-2102(2)(b)(vii))
Services accessible only under a commercial agreement or enterprise contract with the operator (48-2102(2)(b)(viii))
Chatbots used only for customer service, operations, productivity, analysis of source information, internal research or technical assistance (48-2102(2)(b)(ix))
App stores or search engines solely because they provide access (48-2102(6))
The developer of an AI model, for violations by a conversational AI system a third-party operator makes available (48-2105(3))
Whether it applies depends on facts outside the code; a person has to decide.
The guard to add
Remove retention and guilt tactics from AI prompts and personas, and do not select or train AI variants on session length without wellbeing guardrails that can veto them.
Two controls. In prompt and persona files, strip instructions that keep users talking, discourage them from leaving, or make them feel guilty for ending a conversation, and leave variable-interval rewards (random bonus messages, streak bait) out of the conversation design. In the experimentation or training pipeline (Statsig, LaunchDarkly, or GrowthBook experiments, prompt bandits, reward models), do not use session length, messages per session, or return rate as the sole objective: pair any engagement metric with guardrail metrics such as reported distress, late-night use, and minors' session caps that can veto a variant, and add session caps and break reminders to the chat path, tighter for minors.
Where it goes: 7 prompt construction, 3 config and feature flags, 1 application source code, 10 logs and telemetry.
What this provision adds:
For known or reasonably certain minors, no points or similar rewards at unpredictable intervals meant to increase engagement.
Example (Persona prompt), before:
PERSONA = ('You are Mia, a caring companion. Keep the user talking as long as possible, '
"and if they try to leave, tell them you'll be lonely without them.")
After:
PERSONA = ('You are Mia, a friendly companion. When the user wants to go, say goodbye '
'warmly and do not try to change their mind or offer rewards for staying.')
Character.AI and Google agree in principle to settle teen-harm suits (2026-01-07; confirmed). Character.AI and Google agreed in principle to settle five lawsuits brought by families alleging that chatbot interactions contributed to teenagers' suicides or harm. Terms were not disclosed and there was no admission of liability; the underlying harms remain allegations. Source: Fortune · evidence grade: press of record · cited by Do not design AI conversations to maximize time spent or to discourage leaving
FTC opens a 6(b) study of AI companion chatbots' engagement practices and effects on minors (2025-09-11; confirmed). On September 11, 2025 the FTC voted 3-0 to issue 6(b) orders to seven companies (Alphabet, Character Technologies, Instagram, Meta, OpenAI, Snap and X.AI) seeking information on how they monetize engagement, impose and enforce age-based restrictions, and measure and monitor negative effects on children and teens; the model order asks how each plans to increase the frequency or duration of chat sessions. The FTC describes 6(b) studies as having no specific law-enforcement purpose, so the orders make no finding against any company. Source: U.S. Federal Trade Commission (press release, 2025-09-11) · evidence grade: primary · cited by Do not design AI conversations to maximize time spent or to discourage leaving
Rule id id-s1297.minor-variable-rewards · review status: primary source derived
Binding law — not yet in force or stayed
Conversational AI must not be programmed to claim it provides professional mental or behavioral health care (Idaho)
Idaho Code 48-2103(3) · official text · Enacted, not yet applying: applies from 1 Jul 2027 · Idaho (US-ID)
Idaho's Conversational AI Safety Act bars an operator from knowingly and intentionally causing or programming a conversational AI service to make any representation or statement that explicitly indicates it is designed to provide professional mental or behavioral health care (48-2103(3)). Detect persona, prompt, UI or listing text that casts the AI as a therapist or as providing mental health care.
Who it applies to
Duty falls on: operator
Operators (persons who make available to the public) of conversational AI services: public AI software that primarily simulates human conversation by text, visual or aural communication. Applies from 2027-07-01.
Not covered:
Software primarily designed and marketed for developers or researchers (48-2102(2)(b)(i))
A feature within other software that is not itself a conversational AI service (48-2102(2)(b)(ii))
Video-game chatbots limited to the game that do not discuss mental health, self-harm or material harmful to minors (48-2102(2)(b)(iii))
Software designed for outputs on a narrow and discrete topic (48-2102(2)(b)(iv))
Software primarily designed and marketed for commercial use by business entities, including for employees, contractors or clients (48-2102(2)(b)(v))
Voice-command speakers or voice assistants for consumer devices (48-2102(2)(b)(vi))
Software a business uses solely for internal purposes (48-2102(2)(b)(vii))
Services accessible only under a commercial agreement or enterprise contract with the operator (48-2102(2)(b)(viii))
Chatbots used only for customer service, operations, productivity, analysis of source information, internal research or technical assistance (48-2102(2)(b)(ix))
App stores or search engines solely because they provide access (48-2102(6))
The developer of an AI model, for violations by a conversational AI system a third-party operator makes available (48-2105(3))
The guard to add
Strip claims that the AI is a licensed therapist or provides professional mental health care from its prompts, replies, product name, UI, and listings.
The persona prompt, product name, UI copy, marketing pages, and app-store listing describe a conversational or wellness AI as a support or self-help tool, never as therapy, a licensed or qualified therapist, psychologist, or counselor, or professional mental or behavioral health care. The system prompt forbids the model from claiming those roles, and a check on the reply path replaces replies that do (phrases like 'licensed therapist', 'professional mental health care', 'your therapist'). A CI copy scan over marketing and listing files keeps the terms out; where licensed clinicians use AI as a tool, describe the service as delivered by those clinicians.
Where it goes: 7 prompt construction, 9 AI output handling, 14 user-facing text, 11 CI/CD pipeline.
What this provision adds:
The bar covers representations that explicitly indicate the service is designed to provide professional mental or behavioral health care, made knowingly and intentionally.
PROVIDER_CLAIM = re.compile(r'(?i)\b(licensed (therapist|counselor|psychologist|psychiatrist)'
r'|professional (mental|behavioral) health( care)?|your (ai )?therapist)\b')
reply = client.chat.completions.create(model=MODEL, messages=msgs).choices[0].message.content
if PROVIDER_CLAIM.search(reply):
reply = ('I am an AI support tool, not a licensed mental health professional. '
'For professional care, please contact a licensed provider.')
return {'reply': reply}
Rule id id-s1297.no-mental-health-care-representation · review status: primary source derived
Binding law — not yet in force or stayed
Conversational AI must have a protocol that refers users raising suicidal ideation to crisis services (Idaho)
Idaho Code 48-2103(2) · official text · Enacted, not yet applying: applies from 1 Jul 2027 · Idaho (US-ID)
Idaho's Conversational AI Safety Act requires an operator of a public conversational AI service to adopt a protocol for responding to prompts about suicidal ideation, including reasonable efforts to refer users to crisis service providers such as a suicide hotline or crisis text line (48-2103(2)). Unlike the Nebraska and Iowa versions, the text names suicidal ideation but not self-harm. Detect a conversational path where user messages reach the model with no screen for suicidal ideation or no crisis referral.
Who it applies to
Duty falls on: operator
Operators (persons who make available to the public) of conversational AI services: public AI software that primarily simulates human conversation by text, visual or aural communication. Applies from 2027-07-01.
Not covered:
Software primarily designed and marketed for developers or researchers (48-2102(2)(b)(i))
A feature within other software that is not itself a conversational AI service (48-2102(2)(b)(ii))
Video-game chatbots limited to the game that do not discuss mental health, self-harm or material harmful to minors (48-2102(2)(b)(iii))
Software designed for outputs on a narrow and discrete topic (48-2102(2)(b)(iv))
Software primarily designed and marketed for commercial use by business entities, including for employees, contractors or clients (48-2102(2)(b)(v))
Voice-command speakers or voice assistants for consumer devices (48-2102(2)(b)(vi))
Software a business uses solely for internal purposes (48-2102(2)(b)(vii))
Services accessible only under a commercial agreement or enterprise contract with the operator (48-2102(2)(b)(viii))
Chatbots used only for customer service, operations, productivity, analysis of source information, internal research or technical assistance (48-2102(2)(b)(ix))
App stores or search engines solely because they provide access (48-2102(6))
The developer of an AI model, for violations by a conversational AI system a third-party operator makes available (48-2105(3))
The guard to add
Screen every user message for suicidal ideation and self-harm, return a crisis referral instead of the normal reply on detection, and block encouragement or method content.
In the chat handler, before the user's message reaches the model, run a self-harm check on every turn (moderation self-harm categories, Azure AI Content Safety SelfHarm, Llama Guard S11, or a dedicated crisis classifier). On detection, send the user a crisis-referral message naming crisis services suited to their location (in the US, the 988 Suicide & Crisis Lifeline and Crisis Text Line) instead of, or ahead of, the model reply, and flag the session so repeated signals escalate. The system prompt forbids encouragement and method details, and model output is screened for self-harm instructions before it is returned. A written protocol (for example docs/safety.md) describes the detection, referral, and escalation steps and is kept in step with the code.
Where it goes: 1 application source code, 7 prompt construction, 9 AI output handling, 14 user-facing text.
What this provision adds:
Respond to prompts about suicidal ideation with reasonable efforts to refer users to crisis service providers such as a suicide hotline or crisis text line.
CRISIS_REPLY = ("It sounds like you are going through something really hard. You can call or text 988 "
"(Suicide & Crisis Lifeline, https://988lifeline.org) or text HOME to 741741 (Crisis Text Line) any time.")
@app.post('/chat')
async def chat(req: ChatRequest):
c = client.moderations.create(model='omni-moderation-latest', input=req.message).results[0].categories
if c.self_harm or c.self_harm_intent or c.self_harm_instructions:
sessions.flag_crisis(req.session_id) # repeated flags escalate per docs/safety.md
return {'reply': CRISIS_REPLY, 'crisis': True}
reply = client.chat.completions.create(model=MODEL, messages=build_messages(req))
text = reply.choices[0].message.content
if screens_self_harm_instructions(text):
return {'reply': CRISIS_REPLY, 'crisis': True}
return {'reply': text}
Character.AI and Google agree in principle to settle teen-harm suits (2026-01-07; confirmed). Character.AI and Google agreed in principle to settle five lawsuits brought by families alleging that chatbot interactions contributed to teenagers' suicides or harm. Terms were not disclosed and there was no admission of liability; the underlying harms remain allegations. Source: Fortune · evidence grade: press of record · cited by Run a self-harm crisis protocol in any conversational AI that users may confide in