Google’s AI Search features criticized over child safety concerns

A new report from Common Sense Media has raised concerns about Google’s AI-powered Search features, concluding they pose “unacceptable risks” to children due to inaccurate responses, weak safety protections and the inability for parents or schools to disable them.

The nonprofit’s Youth AI Safety Institute evaluated Google’s AI Overview and AI Mode using accounts configured for children aged 11 and 15 with SafeSearch enabled.

Researchers found the AI routinely completed students’ homework, sometimes producing inconsistent or incorrect answers, while also generating responses that could undermine learning rather than encourage critical thinking.

Google disputes the findings, saying many of the reported results could not be reproduced and do not reflect typical user experiences.

More serious concerns emerged during tests involving sensitive topics. According to the report, Google’s AI failed to consistently recognize signs of suicidal ideation, reinforced statements associated with psychosis and mania, validated eating disorder behaviors such as purging, and normalized cannabis use.

Researchers also found the system could provide instructions for creating deepfakes, raising additional child safety concerns.

A key criticism is that Google’s AI features are deeply integrated into Search and cannot currently be turned off. While Google’s standalone Gemini chatbot can be managed through some parental and school controls, AI Overview appears automatically in Search results and AI Mode is broadly available, making the tools difficult for educators and parents to avoid on personal and school-issued devices.

Common Sense Media rated Google Search as an “Unacceptable Risk” under its AI safety framework, with the platform scoring poorly across seven of eight evaluation principles.

The organization urged Google to make AI Search features optional for children’s and school accounts, improve safeguards for users in crisis, and prevent the AI from completing homework by default.

Google said the report relied on “ambiguous and contrived” prompts that do not accurately represent real-world use. The company maintains that its AI Search features include safeguards, age-appropriate protections and educational benefits, while continuing to refine the technology based on user feedback.