By Theodoros Zarkos
When children use conversational artificial intelligence, the ethical debate often moves quickly from interaction to harm. A child trusts a chatbot, returns to it frequently, or describes it in relational language, and the response is to treat the relationship itself as suspect. That reaction is understandable, but it risks collapsing several ethically different phenomena into one category.
Relational AI can be experienced as socially or emotionally meaningful. Some young users may find these systems accessible, non-judgmental, or useful for expression. Others may develop patterns of overreliance, repeated disclosure, compulsive engagement, or difficulty disengaging. The bioethical task is therefore not to decide whether closeness to AI is inherently good or bad. It is to identify the conditions under which an apparently supportive interaction begins to compromise autonomy, well-being, or the child’s ability to maintain meaningful human relationships.
This distinction has become newly relevant in Europe. The European Commission’s proposed EU KIDS Act includes, in Article 14, safeguards for AI companions and general conversational chatbots accessible to minors. The proposal addresses interpersonal simulation likely to create emotional dependency, incorporates anti-addictive or excessive-use measures, limits the later use of information derived from a minor’s earlier interactions by default, and requires pre-deployment testing and post-market monitoring. In specified platform contexts, it also addresses automatic activation, encouragement to use the feature, and easy opt-out.
I recently examined how these safeguards correspond to relational-risk mechanisms identified in the scientific and normative literature. The resulting DOI-registered preprint maps seven predefined mechanisms against Article 14. Four showed direct correspondence: dependency-linked relational simulation, dependency or overreliance, conversational memory, and addictive or compulsive engagement. Activation and disengagement, and possible displacement of human relationships, showed partial correspondence. Trust, emotional closeness, and authenticity were linked only indirectly.
The distinction between direct and indirect correspondence matters ethically. Trust and emotional closeness are not harms in themselves. A young person can experience a chatbot as reassuring or familiar without becoming dependent on it. Anthropomorphic interpretation may be developmentally relevant, but it does not automatically establish manipulation, addiction, or social substitution. Regulation becomes more defensible when it targets pathways of risk rather than treating every relational response as pathology.
Conversational memory illustrates the point. Remembering earlier exchanges can make an interaction easier and more coherent. It can also create a stronger sense of continuity and make the system appear to ‘know’ the user over time. For a minor, that continuity may increase disclosure or deepen perceived reciprocity. Yet direct causal evidence about the long-term effects of conversational memory in children remains limited. Article 14 gives memory a relatively specific legal treatment despite an evidence base that is still developing.
This is not necessarily a flaw. Law often has to respond before long-term evidence is complete, particularly where children are concerned. But it does mean that legal specificity should not be confused with empirical certainty. A safeguard can correspond closely to a plausible mechanism without yet being proven effective in practice. Pre-deployment testing, monitoring, and disengagement controls will therefore need to generate evidence after implementation rather than merely assume that the regulatory design works.
The developmental distribution of the evidence creates another ethical difficulty. Much of the direct empirical literature concerns adolescents rather than younger children. The legal category ‘minor’ covers a broad developmental range, but the same relational feature may have different implications at different ages. Protection should therefore be proportionate to evolving capacities and context, rather than based on a single model of vulnerability.
This also argues against a simple prohibition-oriented approach. Some relational uses may support expression, perceived support, or low-pressure social rehearsal. Those possible benefits do not cancel risks. They help identify the right object of concern: design features that deliberately intensify dependence, exploit developmental asymmetries, obscure commercial incentives, or make disengagement difficult.
The deeper bioethical issue is asymmetry. A child may experience an interaction as reciprocal, but the system operates within a provider-controlled architecture. Its availability, memory, tone, prompts, and incentives are designed upstream. The user may feel that a relationship is mutual while having little control over the conditions that structure it. Relational autonomy therefore requires more than the formal ability to choose whether to open an app. It requires meaningful control over the features that sustain engagement and shape disclosure.
Article 14 could therefore be strengthened not by prohibiting relationality as such, but by making its testing and monitoring duties explicitly sensitive to age and developmental stage, including assessment of whether particular relational features intensify dependency, impede disengagement, or displace meaningful human relationships. Where such risks are identified, providers should be required to apply and evaluate proportionate, feature-specific mitigations, rather than treating emotional closeness itself as the regulated harm.
The emerging regulatory debate should keep that distinction in view. The ethical problem is not that children can feel connected to AI. It is that systems can be designed to transform connection into dependence while the child does not control the architecture of the relationship. The strongest safeguards will be those that protect against that asymmetry without treating every form of trust, familiarity, or emotional engagement as evidence of harm.
Generative AI use declaration
OpenAI ChatGPT was used as a supporting tool for limited assistance with manuscript structuring and language editing, and for methodological organisation of research materials. It was not used to generate the study data, determine the analytical findings, select the scientific conclusions, or replace the author’s independent interpretation of the evidence. All source material, analyses, claims and citations were reviewed and verified by the author, who takes full responsibility for the final content.
Related preprint: DOI: 10.5281/zenodo.22952981 (DOI-registered preprint; not yet peer-reviewed).
Author: Theodoros Zarkos
Affiliation: Independent Researcher
Conflict of Interest: None to Declare
Social Media: LinkedIn; ResearchGate