Following a slew of reports of ChatGPT users experiencing mental health crises, OpenAI said it's making a version just for teens.
“We prioritize safety ahead of privacy and freedom for teens,” OpenAI CEO Sam Altman said in a blog post. “This is a new and powerful technology, and we believe minors need significant protection," he added.
The update to ChatGPT coming later this month will help it identify whether someone is under the age of 18. If a minor is identified, they will be “automatically” directed to a version of ChatGPT that has “age-appropriate policies,” according to OpenAI. Altman emphasized in his post that ChatGPT is for those above the age of 13.
“If there is doubt, we’ll play it safe and default to the under-18 experience,” Altman said. “In some cases or countries we may also ask for an ID; we know this is a privacy compromise for adults but believe it is a worthy tradeoff.”
OpenAI said it will give adults “ways to prove their age” to move back to the adult ChatGPT experience. It did not elaborate on how adults might be able to do so.
In the version of ChatGPT for teens, OpenAI said it will block graphic sexual content and, when a user is exhibiting signs of “acute distress,” it will “potentially” involve law enforcement.
Altman added that the teen-version of ChatGPT will be trained to avoid conversations that are flirtatious or about suicide or self-harm — “even in a creative writing setting.”
Researchers have documented how easy it is to circumvent limits set by chatbot companies. Age-verification rules are also known to be easily bypassed.
As for the adult-version of the chatbot, Altman emphasized that it won’t impede on users’ “freedom” while staying within “very broad bounds of safety.” Adults will still be able to engage in "flirtatious talk” if they ask for it. Altman added that ChatGPT “should not provide instructions” on “how to commit suicide,” but added a caveat that if a user is “asking for help writing a fictional story that depicts a suicide” then the chatbot “should help with that request.”
"Treat our adult users like adults’ is how we talk about this internally,” Altman said.
Parental controls
OpenAI first announced parental controls at the end of August. They will be available at the end of September and will let parents link to their teen’s account, guide how ChatGPT responds, disable features, notify them when their teen is in “acute distress,” and set times when their teen can’t use ChatGPT.
The AI company made the announcement on the same day it was hit with a lawsuit after a teenager died by suicide with the alleged help of ChatGPT. Altman was also named in the suit.
At the time, OpenAI said the “recent heartbreaking cases” of users leaning on ChatGPT in the “midst of acute crises weigh heavily on us.”
In addition to teens, there have been numerous cases of adults having mental health crises linked to reported encouragement from ChatGPT. Some have started calling these cases of AI chatbots encouraging mental health episodes as “AI psychosis” — or “ChatGPT psychosis."
OpenAI's August announcement stated other safety measures that would impact adults as well as teens, including adding emergency resources. Altman’s more recent statement on the matter, however emphasized the importance of “freedom” for adult users.
Advocates want government action
OpenAI’s latest safety notice came hours before a Senate subcommittee on crime and counterterrorism hearing on the harm of AI chatbots.
The subcommittee heard from safety advocates and parents of teens who died by suicide from alleged encouragement or assistance from chatbots — including, in one case, ChatGPT.
Advocates gave recommendations to the committee on what actions the government should take to protect teens from AI — many going much further than what OpenAI intends to do.
“AI systems lack the clinical training and the diagnostic capabilities to safely chat with teens about mental health. They’re programmed to maintain engagements, not prioritize safety," Robbie Torney, senior director of AI programs at Common Sense Media, said to the committee.
"We need decisive action," he continued. "Common Sense Media recommends that Congress require companies implement robust age assurance and limit AI companion access for users under 18, establish liability frameworks to hold platforms accountable when their AI systems harm children, mandate safety testing and transparent reporting of AI failures particularly for platforms accessible to minors, and protects states’ rights to develop their own AI policies.”
Another advocate, Mitch Prinstein, chief of psychology strategy and integration at the American Psychological Association, remarked on the legal agreements users sign when signing up for a chatbot.
"Have you read all of the legal language that platforms ask us to agree to?" he said. "Even if teens wanted to, it's not written in a way for adolescents to understand nor are they capable of appreciating the long-term risks that they face when yielding their lifetime rights to their data.”
Prinstein told the Congressional committee that it "must enact comprehensive data privacy legislation that establishes privacy protection as the default setting for minors and explicitly prohibits their sale of data,” adding that the APA urges Congress to "prohibit AI from misrepresenting itself as psychologists or therapists and to mandate clear and persistent disclosure that users are interacting with an AI bot.”