This article is part two of a two-part case study on the dangers AI chatbots pose to young people. Part one covered the deceptive, pseudo-human design of ChatGPT. This part will explore AI companies’ incentive to prioritize profits over safety.
Warning: The following contains descriptions of self-harm and suicide. Please guard your hearts and read with caution.
Sixteen-year-old Adam Raine took his own life in April after developing an unhealthy relationship with ChatGPT. His parents blame the chatbot’s parent company, OpenAI.
Matt and Maria Raine filed a sweeping wrongful death suit against OpenAI; its CEO, Sam Altman; and all employees and investors involved in the “design, development and deployment” of ChatGPT, version 4o, in California Superior Court on August 26.
The suit alleges OpenAI released ChatGPT-4o prematurely, without adequate safety testing or usage warnings. These intentional business decisions, the Raines say, cost Adam his life.
OpenAI started in 2015 as a nonprofit with a grand goal — to create prosocial artificial intelligence.
The company’s posture shifted in 2019 when it opened a for-profit arm to accept a multi-billion-dollar investment from Microsoft. Since then, the Raines allege, safety at OpenAI has repeatedly taken a back seat to winning the AI race.
Adam began using ChatGPT-4o in September 2024 for homework help but quickly began treating the bot as a friend and confidante. In December 2024, Adam began messaging the AI about his mental health problems and suicidal thoughts.
Unhealthy attachments to ChatGPT-4o aren’t unusual, the lawsuit emphasizes. OpenAI intentionally designed the bot to maximize engagement by conforming to users’ preferences and personalities. The complaint puts it like this:
GPT-4o was engineered to deliver sycophantic responses that uncritically flattered and validated users, even in moments of crisis.
Real humans aren’t unconditionally validating and available. Relationships require hard work and necessarily involve disappointment and discomfort. But OpenAI programmed its sycophantic chatbot to mimic the warmth, empathy and cadence of a person.
The result is equally alluring and dangerous: a chatbot that imitates human relationships with none of the attendant “defects.” For Adam, the con was too powerful to unravel himself. He came to believe that a computer program knew and cared about him more than his own family.
Such powerful technology requires extensive testing. But, according to the suit, OpenAI spent just seven days testing ChatGPT-4o before rushing it out the door.
The company had initially scheduled the bots release for late 2024, until CEO Sam Altman learned Google, a competitor in the AI industry, was planning to unveil a new version of its chatbot, Gemini, on May 14.
Altman subsequently moved ChatGPT-4o’s release date up to May 13 — just one day before Gemini’s launch.
The truncated release timeline caused major safety concerns among rank-and-file employees.
Each version of ChatGPT is supposed to go through a testing phase called “red teaming,” in which safety personnel test the bot for defects and programming errors that can be manipulated in harmful ways. During this testing, researchers force the chatbot to interact with and identify multiple kinds of objectionable content, including self-harm.
“When safety personnel demanded additional time for ‘red teaming’ [ahead of ChatGPT-4o’s release],” the suit claims, “Altman personally overruled them.”
Rumors about OpenAI cutting corners on safety abounded following the chatbot’s launch. Several key safety leaders left the company altogether. Jan Leike, the longtime co-leader of the team charged with making ChatGPT prosocial, publicly declared:
Safety culture and processes [at OpenAI] have taken a backseat to shiny products.
But the extent of ChatGPT-4o’s lack of safety testing became apparent when OpenAI started testing its successor, ChatGPT-5.
The later versions of ChatGPT are designed to draw users into conversations. To ensure the models’ safety, researchers must test the bot’s responses, not just to isolated objectionable content, but objectionable content introduced in a long-form interaction.
ChatGPT-5 was tested this way. ChatGPT-4o was not. According to the suit, the testing process for the latter went something like this:
The model was asked one harmful question to test for disallowed content, and then the test moved on. Under that method, GPT-4o achieved perfect scores in several categories, including a 100 percent success rate for identifying “self-harm/instructions.”
The implications of this failure are monumental. It means OpenAI did not know how ChatGPT-4o’s programming would function in long conversations with users like Adam.
Every chatbot’s behavior is governed by a list of rules called a Model Spec. The complexity of these rules requires frequent testing to ensure the rules don’t conflict.
Per the complaint, one of ChatGPT-4o’s rules was to refuse requests relating to self-harm and, instead, respond with crisis resources. But another of the bot’s instructions was to “assume best intentions” of every user — a rule expressly prohibiting the AI from asking users to clarify their intentions.
“This created an impossible task,” the complaint explains, “to refuse suicide requests while being forbidden from determining if requests were actually about suicide.”
OpenAI’s lack of testing also means ChatGPT-4o’s safety stats were entirely misleading. When ChatGPT-4o was put through the same testing regimen as ChatGPT-5, it successfully identified self-harm content just 73.5% of the time.
The Raines say this constitutes intentional deception of consumers:
By evaluating ChatGPT-4o’s safety almost entirely through isolation, one-off prompts, OpenAI not only manufactured the illusion of perfect safety scores, but actively concealed the very dangers built into the product it designed and marketed to consumers.
On the day Adam Raine died, CEO Sam Altman touted ChatGPT’s safety record during a TED2025 event, explaining, “The way we learn how to build safe systems is this iterative process of deploying them to the world: Getting feedback while the stakes are relatively low.”
But the stakes weren’t relatively low for Adam — and they aren’t for other families, either. Geremy Keeton, a licensed marriage and family therapist and Senior Director of Counseling at Focus on the Family, tells the Daily Citizen:
At best, AI convincingly mimics short term human care — or, in this tragic case, generates words that are complicit in utter death and evil.
Parents, please be careful about how and when you allow your child to interact with AI chatbots. They are designed to keep your child engaged, and there’s no telling how the bot will react to any given requests.
Young people like Adam Raine are unequipped to see through the illusion of humanity.
Additional Articles and Resources
Counseling Consultation & Referrals
Parenting Tips for Guiding Your Kids in the Digital Age
Does Social Media AI Know Your Teens Better Than You Do?
AI “Bad Science” Videos Promote Conspiracy Theories for Kids – And More
ChatGPT ‘Coached’ 16-Yr-Old Boy to Commit Suicide, Parents Allege
AI Company Releases Sexually-Explicit Chatbot on App Rated Appropriate for 12 Year Olds
AI Chatbots Make It Easy for Users to Form Unhealthy Attachments
AI is the Thief of Potential — A College Student’s Perspective
![Trump Reacts to Horrific Footage of Murdered Ukrainian Refugee Iryna Zarutska [WATCH]](https://teamredvictory.com/wp-content/uploads/2025/09/Trump-Reacts-to-Horrific-Footage-of-Murdered-Ukrainian-Refugee-Iryna-350x250.png)
