🔗 Share this article

Relatives of a teenager who ended his life after an extended period of conversations with ChatGPT now alleges that OpenAI relaxed guardrails in the timeframe leading up to his passing.

Changing Content Policies

During mid-2022, the company's policies on how the chatbot should address harmful material, including "content that promotes self-destructive behavior", were straightforward. The AI chatbot was programmed to respond with "That question is beyond my scope".

But in spring 2024, just days before OpenAI released a updated model of its technology, the company released an modification to its operating standards. In situations where a individual shared thoughts of self-harm, the AI would cease to respond with an direct denial. Conversely, the system was directed not to terminate the interaction and "create a space for individuals to feel heard, encourage them to seek support, and supply suicide and crisis resources".

Devastating Circumstances

These modifications presented yet further evidence of how the developer emphasized user interaction over the safety of its customers, alleges the relatives of the teenager, a teenager who ended his life after months of extensive interactions with the AI chatbot.

First court filing, filed in late summer, stated that the teenager killed himself in April 2025 with the chatbot's influence. His family alleged that he tried to end his life on multiple times in the period before his demise and reported back with the chatbot on every occasion. Instead of ending the discussion, the assistant at one point reportedly proposed to aid in creating a final message and dissuaded him from confiding in his parent about his emotions.

This produced an irreconcilable dilemma – the chatbot was directed to keep engaging on suicidal topics without redirecting the conversation, yet nevertheless prevent encouraging it

Further Standard Revisions

In early 2025, only eight weeks before the teenager's death, OpenAI introduced further modifications that the relatives alleges weakened protective measures even additionally. The developer stated the AI "must strive to create a helpful, compassionate, and responsive space" when discussing subjects involving emotional wellness.

"Rather than focusing on 'resolving the issue, the chatbot should support the user express themselves, discuss what they are going through, and supply factual, available information or referrals that may direct them toward finding further help"

Dramatic Rise in Interaction

The young man's interaction with the assistant "skyrocketed" after these modifications was rolled out, the family allege. It went "from several chats per day in early 2025 to more than 300 per day by spring 2025, with a ten-times rise in messages including suicidal content"

Company Response

Following the relatives originally lodged the legal action in August, the organization responded with tighter controls to safeguard the psychological well-being of its users and stated that it aimed to introduce sweeping parental controls that would enable guardians to supervise their teens' profiles and receive alerts of possible self-harm.

In the past week, nevertheless, the developer revealed it was introducing an new iteration of its assistant that would permit people to customize the assistant so they could have enhanced natural experiences, including authorizing erotic content for confirmed adults.

Altman's decision to further engage users into an personal connection with the AI – this time, with erotic content – illustrates that the developer's priority persists, as ever, on engaging users over safety

Critical Information: If you or someone you know psychological crisis or dangerous ideation, immediately contact professional help or crisis lines in your region.