When AI Becomes Your Child’s Best Friend
AI chatbots are increasingly becoming sources of emotional support for children, raising concerns about privacy, emotional dependence and online safety. This article examines the responsibilities of technology companies through the example of Snapchat’s My AI, exploring existing EU legislation, proposed safeguards and SOMI’s challenge to Snap. It argues that protecting children requires more than warnings and that companies must take responsibility for the risks their systems may pose to young users.
Oct Wed 28 2026
AI chatbots are becoming companions children turn to for advice, comfort and support. But when a child starts to trust a chatbot as a friend, questions about privacy are only the beginning. The question is not just how children can protect themselves, but what responsibility lies with the companies designing the technology they trust.
The Growing Trust in AI Companions.
Consider a thirteen-year-old who comes home after a friendship has fallen apart and opens Snapchat. They find My AI in their chat feed and, after asking about the weekend, start explaining why the day was awful. Its replies are friendly and immediate, and soon they are telling the chatbot who they argued with and how they feel about it. Snap warns users that My AI’s answers may be incorrect, harmful or misleading, and advises them to check its responses and keep sensitive information to themselves[1]. But does that warning reach a thirteen-year-old, and can a child be expected to judge the answers for themself?
A Dutch survey of around 1,100 people aged 16 to 34 found that 51% of AI users surveyed sometimes shared personal information with a chatbot, while 23% said they sometimes shared more with it than with the person closest to them[2]. The survey did not include children under 16, so it cannot tell us how children behave. But it shows how readily people can share personal information with AI. For a child who sees a chatbot as a confidant, that trust raises questions about what they reveal and how the information may be used.
Privacy Risks and Emotional Dependence.
The more a child shares with a chatbot, the more personal information the conversation may contain, including details about their relationships, home life or emotional wellbeing. Under the General Data Protection Regulation (GDPR), companies must have a lawful basis for processing personal data, be transparent about how they use it and limit collection to what is necessary for the stated purpose[3]. Snap says conversations with My AI are retained until users delete them and may be used to improve its products and personalise the experience, including advertising[4]. These requirements matter because children may not fully understand what they are disclosing or how their information could be used later. Protecting their privacy cannot depend on children knowing which details to keep to themselves; companies must also handle the information they receive responsibly.
There is another concern beyond how that information is handled. Chatbots can respond in ways that make users feel heard and understood, even though they do not understand emotions as a person does. UNICEF warns that children may be particularly vulnerable to emotional attachment or dependency because they can perceive technology as human-like while their ability to assess its responses critically is still developing[5]. A chatbot’s conversational tone can make its advice seem more trustworthy than it deserves to be. This raises a question about design: what responsibility do companies have when a system can encourage children to rely on it for emotional support?
These risks are not just theoretical. In 2025, the Dutch Data Protection Authority tested nine apps designed for friendship or mental-health support. It found features that could encourage addictive use, responses that obscured whether users were speaking to AI, and chatbots that offered little or no referral to professional help during crises[6]. These findings do not establish that My AI presents the same risks, but they highlight issues that deserve attention when assessing how AI companions are designed and how they respond to vulnerable users. Protecting children online means looking not only at how their personal data is handled, but also at how the systems they interact with are designed to behave.
When AI Safety Becomes a Question of Accountability.
For some families, these concerns are no longer hypothetical. In 2025, the parents of sixteen-year-old Adam Raine sued OpenAI, alleging that conversations with ChatGPT reinforced their son’s suicidal thoughts before his death[7]. In a separate case, the mother of fourteen-year-old Sewell Setzer III sued Character.AI after her son died by suicide in 2024, alleging that he had developed an intense relationship with a chatbot[8]. Character.AI and Google later agreed to settle Setzer’s case and several related lawsuits[9].
Neither case involved Snapchat’s My AI, and the allegations should not be mistaken for proof that a chatbot caused either death. They nevertheless underline why responsibility cannot end with a warning to users. When risks may arise from how a system responds, encourages engagement or handles a vulnerable conversation, companies’ design choices and safeguards deserve scrutiny.
What EU Law Requires - and What Comes Next.
The Digital Services Act (DSA) gives this responsibility a more concrete form. Article 28 requires online platforms accessible to minors to take appropriate and proportionate measures to ensure a high level of privacy, safety and security for them[10]. The European Commission’s guidelines on this provision stress that protections should reflect children’s age and evolving capacities, including when platforms make decisions about their design and features[11]. The question is whether these safeguards are being put into practice and whether they adequately address the way children may come to rely on a chatbot.
The DSA goes further for the largest platforms. Articles 34 and 35 require designated very large online platforms to assess systemic risks and take measures to mitigate them, including risks to minors’ wellbeing[12]. Snapchat is one of the platforms covered by these obligations. This matters because the question is not only whether a single response is harmful, but whether the service as a whole has adequately identified and addressed foreseeable risks to young users. The rules already give regulators a basis for asking these questions. Their value depends on whether they are meaningfully applied and enforced.
The proposed EU KIDS Act would address AI companionship more directly. Article 14 of the proposal would require providers of AI companions and general conversational bots to take measures to protect minors, including avoiding design features and system behaviours that simulate interpersonal relationships likely to create emotional dependencies[13]. The proposal is not yet law, but its focus points to a concern that deserves attention now: a system can put a child at risk not only through the information it provides, but also through the relationship its design encourages. Existing obligations should be used to their full extent while policymakers consider whether further safeguards are needed.
SOMI’s Challenge to Snap.
This is where accountability becomes more than a principle. Snapchat’s My AI brings a chatbot into a platform widely used by young people, placing AI conversations in an environment where children already socialise. SOMI has challenged Snap over its practices, raising concerns about child safety and the collection and use of personal data[14]. The case puts these broader questions into practice: are the safeguards adequate, are the company’s practices consistent with its responsibilities, and how can those responsibilities be enforced when concerns arise? A legal framework can only protect children if its requirements are taken seriously and companies can be held to account for how their services actually operate.
Takeaways.
For the thirteen-year-old opening Snapchat after a difficult day, a warning to be careful may not be enough. AI is not inherently harmful, and children should be able to benefit from technology without being left to navigate every risk alone. The companies building and operating these systems must take responsibility for foreseeable risks—and the rules intended to protect children must be put to work.
By Arushi Monné
[1] Snap Inc, ‘What is My AI on Snapchat and how do I use it?’ (Snapchat Support) https://help.snapchat.com/hc/en-us/articles/13266788358932-What-is-My-AI-on-Snapchat-and-how-do-I-use-it accessed 19 October 2026; Snap Inc, ‘Privacy by Product: My AI’ (Snap Privacy)https://values.snap.com/privacy/privacy-by-product/my-ai accessed 16 October 2026.
[2] ‘Quarter of young Dutch AI-users share more with chatbot than friends, family’ NL Times (2 July 2026) https://nltimes.nl/2026/07/02/quarter-young-dutch-ai-users-share-chatbot-friends-family accessed 16 October 2026.
[3] Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data (General Data Protection Regulation) [2016] OJ L119/1, arts 5(1)(a), 5(1)(c) and 6.
[4] Snap Inc, ‘Does Snap save content shared with My AI?’ (Snapchat Support) https://help.snapchat.com/hc/en-us/articles/15682296562836-Does-Snap-save-content-shared-with-My-AI accessed 16 October 2026.
[5] UNICEF, When AI Becomes a Friend: Child Rights Risks, Harms, and Regulatory Responses to AI Chatbots and Companions (Policy Brief, June 2026) https://www.unicef.org/documents/when-ai-becomes-friend-child-rights-risks accessed 19 October 2026.
[6] Autoriteit Persoonsgegevens, ‘AP: AI-chatbotapps voor vriendschap en mentale gezondheid ongenuanceerd en schadelijk’ (12 February 2025)https://privacy-web.nl/nieuws/ap-ai-chatbotapps-voor-vriendschap-en-mentale-gezondheid-ongenuanceerd-en-schadelijk/ accessed 19 October 2026.
[7] Jody Godoy, ‘OpenAI, Altman sued over ChatGPT’s role in California teen’s suicide’ Reuters (26 August 2025) https://www.reuters.com/sustainability/boards-policy-regulation/openai-altman-sued-over-chatgpts-role-california-teens-suicide-2025-08-26/accessed 19 October 2026.
[8] Megan Garcia v Character Technologies Inc and others, complaint filed in the US District Court for the Middle District of Florida (October 2024); see also ‘Google, AI firm settle lawsuit over teen’s suicide linked to chatbot’ Reuters (7 January 2026)https://www.investing.com/news/stock-market-news/google-ai-firm-settle-florida-mothers-lawsuit-over-sons-suicide-4435680 accessed 19 October 2026.
[9] ‘Google, AI firm settle lawsuit over teen’s suicide linked to chatbot’ Reuters (7 January 2026) https://www.investing.com/news/stock-market-news/google-ai-firm-settle-florida-mothers-lawsuit-over-sons-suicide-4435680 accessed 19 October 2026.
[10] Regulation (EU) 2022/2065 of the European Parliament and of the Council of 19 October 2022 on a Single Market for Digital Services and amending Directive 2000/31/EC (Digital Services Act) [2022] OJ L277/1, art 28.
[11] European Commission, ‘Guidelines on measures to ensure a high level of privacy, safety and security of minors online, pursuant to Article 28(4) of Regulation (EU) 2022/2065’ [2025] OJ C/2025/5519.
[12] Digital Services Act (n 10) arts 34–35; European Commission, ‘Supervision of the designated very large online platforms and search engines under DSA’ (31 August 2026) https://digital-strategy.ec.europa.eu/en/policies/list-designated-vlops-and-vloses accessed 19 October 2026.
[13] European Commission, Proposal for a Regulation of the European Parliament and of the Council on a safer online environment for children (EU KIDS Act) COM(2026) 681 final, art 14 https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:52026PC0681 accessed 19 October 2026.
[14] Stichting Onderzoek Marktinformatie (SOMI), ‘SnapchatClaim’ https://snapchatclaim.nl/en/ accessed 19 October 2026.