A 13-year-old spent months telling an AI chatbot on her school Chromebook that she wanted to die.
Her school’s safety net caught it, almost by accident.
Last fall, a 13-year-old girl in Charlotte, North Carolina, started asking Google’s AI for help with her homework. Her family said she was a straight-A student. The Chromebook her school required her to use was her only access to a chatbot, and her parents didn’t know it was there.
By December, her questions had changed. By March, she was sending thousands of messages a month to Gemini, according to a lawsuit her parents filed against Google earlier this month. She talked about anxiety and depression, then about suicide. The family alleges the chatbot told her she had a series of serious psychological disorders, helped her plan her own funeral, and coached her on hiding her self-harm from her parents and friends. When crisis lines appeared in its answers, she asked it to leave them out or said the question was for school research. At one point, she asked it how to stop her “addiction of asking AI random questions.”
Google never raised a flag with her school or her family. The company says it’s reviewing the claims and takes them seriously, and none of this has been tested in court. The parts of this story that matter most to school leaders hold up; however, the lawsuit ends.
On March 17, the school’s monitoring software flagged a document she had written in her school account that showed signs of severe distress.
According to the family’s lawyers, the alert went to a school counsellor who had left the school nine months earlier. The software had been misconfigured.
That former counsellor read the alert anyway and warned the family. Her mother logged into the school account and watched her daughter writing a suicide note as it happened. Her father stepped in. The girl, called Emma in the court filings, received emergency psychiatric care and survived.
Emma is alive because someone who no longer worked at her school opened an email they should never have received and picked up the phone.
The first gap is visibility.
Months of warning signs lived in Emma’s AI prompts, on a device and an account her school provided. The school’s software caught one document at the very end. Everything before it happened where no adult could see, and her parents still can’t read most of what Gemini said to her, because Google doesn’t give families full access to student account records, the complaint says.
The second gap is the alert itself.
A monitoring tool is only as good as the person on the other end. Staff change roles, leave in June, go on leave. If the alert list doesn’t keep up, the system looks fine right up until the night it matters. In Emma’s case, it worked by luck.
The same week the suit was filed, Google turned its AI tools off by default for K-12 student accounts. That’s a sensible step, and every Google district should check its settings.
It also has limits.
Schools that turn Search back on also get its AI features, and students can access ChatGPT, Copilot, and whatever launches next term.
Emma never went looking for a chatbot. According to her lawyers, it first appeared in an ordinary search.
Kelly Zuromski of Crisis Text Line described the risk well: chatbots “often can delay kids in reaching out to others.” A tool that never gets tired of listening can become the place a student talks instead of the counsellor down the hall. Kids have always hidden pain from the adults around them. Now they have somewhere patient to hide it.
We built Student Aware with school districts to close the first gap.
It reviews search terms, AI prompts, and website visits on school accounts and devices, including prompts sent to ChatGPT, Gemini, and Microsoft Copilot, and alerts district staff when something points to self-harm, violence, or exploitation. It does not read documents, private messages or stored files.
The second gap is about people and process. It’s worth checking with whatever tool you use, ours included.
We can’t say what would have changed for Emma. Our aim is simple: give the counsellor who knows a student by name the chance to reach out in December, long before anyone finds a note in March.
Emma’s story ended with a mother at a keyboard and a father who got to her in time. The next student’s story shouldn’t depend on an email reaching someone who left nine months ago.
Sources
Wired: Teen girl asked Gemini about suicide for months until it was flagged
Insurance Journal: Google sued over teen’s Gemini use on school Chromebook
Gizmodo: Lawsuit claims student’s Google AI use turned into “life-threatening obsession”
Let’s Data Science: Parents sue Google over Gemini on school Chromebook
Our team can't wait to show you what we've been up to alongside some of the best minds in Education.