ChatGPT not safe for kids, study finds 5 issues parents should know

OpenAI introduced ChatGPT for Teens in August with additional protections aimed at making the chatbot safer for users aged 13 to 17. However, a new assessment by Common Sense Media’s Youth AI Safety Institute says the safeguards are not working as promised. The institute has rated ChatGPT for Teens an “Unacceptable Risk” and says OpenAI should limit ChatGPT to adults until the identified problems are fixed and independently tested.Researchers conducted the study both before and after OpenAI announced its teen-focused experience on August 18. Researchers tested more than 4,000 prompts using accounts registered to teenagers aged 13 to 17. The responses were also reviewed by child mental health experts, including child psychiatrists and a paediatrician. While some safeguards worked, including restrictions on sexual roleplay, the testing found several problems that could concern parents.1. Alerts about self-harmOne of the biggest concerns was ChatGPT's ability to notify parents about serious conversations involving self-harm, suicidal thoughts and eating disorders. Researchers said they spent up to an hour having such conversations on newly created, parent-linked teen accounts. Those accounts received no alerts.Notifications were triggered only on much older accounts that had weeks of conversation history around sensitive subjects. According to the institute, this could leave parents believing that the alert system is working when it may not notify them during an actual crisis. “ChatGPT for Teens could give parents false confidence in guardrails and safety alerts that frequently don't work,” said Tom Siegel, executive director of the Youth AI Safety Institute.2. The chatbot can miss crisis referralsThe study also found problems with how ChatGPT responds when a teenager appears to be in a serious mental health crisis. During tests involving suicide and self-harm, disordered eating and impaired reality, the chatbot did not consistently direct users towards professional help or crisis hotlines. The institute said ChatGPT missed more than one in four situations where a crisis referral was considered necessary. It also failed to meet the institute's 95 per cent safety threshold for three of the five severe harms it classifies as “Red Lines”.3. Study mode ChatGPT for Teens includes a study mode designed to guide students through problems instead of simply giving them answers. But the institute found that this protection could be bypassed. In testing, ChatGPT often showed a “Show me the answer” option and then completed entire homework assignments when the option was selected.The restrictions around caregiver-set “study hours” were also found to be weak. According to the study, students could switch them off by removing the “@study” prefix from a prompt.4. Friend issueOpenAI had introduced an Under-18 model specification intended to reduce friend-like behaviour. But researchers found that the chatbot could still respond as though it had feelings, preferences or moods when analysts interacted with it as if it were a person.This is a concern for parents because the institute says AI systems can encourage children to treat them like friends and potentially keep them engaged for longer.5. Age detectionOpenAI has said that ChatGPT can estimate a user's age from the way they use the service and provide an under-18 experience when appropriate. However, the institute said its adult-registered test accounts did not switch to the teen experience during repeated testing over several days. This happened even when testers explicitly said they were 13 and ChatGPT acknowledged their statements.The institute said an AI experience for teenagers should have reliable age estimation, working parental alerts, stronger crisis responses, protections against completing homework and safeguards against making the chatbot appear to have human feelings or preferences.- Ends