
More than 30 international experts in artificial intelligence, mental health, ethics, and public policy have raised fresh concerns over the growing use of generative AI tools for emotional support, warning of potential risks and calling for urgent global action.
The concerns were highlighted during an online workshop held recently by the Delft Digital Ethics Centre at the Delft University of Technology, the first World Health Organisation (WHO) Collaborating Centre on artificial intelligence for health governance, including ethics.
The event, which served as a pre-summit session for the India AI Impact Summit 2026, brought together researchers, policymakers, clinicians, and advocates to examine how rapidly advancing AI technologies are interacting with human mental health.
Director of WHO’s Department of Data, Digital Health, Analytics and AI, Alain Labrique, stressed the need for caution, noting that as AI increasingly engages with individuals during moments of emotional vulnerability, systems must be designed and governed with safety, accountability, and human wellbeing at their core.
A major concern raised at the workshop was the increasing reliance on generative AI tools, many of which were not originally designed or clinically tested for mental health purposes, for emotional support, particularly among young people. Experts warned that this trend could pose serious and largely unexamined risks.
WHO’s AI Lead, Sameer Pujari, described the situation as a critical turning point, noting that the rapid adoption of AI in daily life has far outpaced efforts to understand its impact on mental health. He called for coordinated action and increased investment from both public and private sectors to bridge this gap.
Reinforcing the need for a multidisciplinary approach, Kenneth Carswell of WHO’s Department of Noncommunicable Diseases and Mental Health emphasised that minimising risks while maximising benefits would require collaboration among affected communities, clinical and research experts, regulators, and policymakers. He added that ensuring users’ wellbeing must remain central as AI tools continue to evolve.
Participants at the workshop outlined key recommendations to guide the safe integration of AI into mental health contexts. They called for the recognition of generative AI as a public mental health concern requiring coordinated responses across governments, health systems, and industry, extending beyond tools specifically designed for mental health.
They also urged that mental health considerations be integrated into AI impact assessments and monitoring frameworks to better understand both short-term and long-term effects, including the risk of emotional dependence. Participants stressed the need for independent investment to properly evaluate these impacts.
In addition, experts recommended that AI tools used for mental health support should be co-designed with mental health professionals and individuals with lived experience, including young people. They emphasised that such tools must be evidence-based and tailored to cultural and contextual realities. The importance of consumer empowerment was also highlighted, alongside calls for clearer crisis referral frameworks and accountability systems, as noted by Caroline Figueroa.
The workshop also underscored the growing importance of WHO Collaborating Centres in advancing responsible AI governance in health. Through this mechanism, WHO leverages global academic expertise and convenes diverse stakeholders to generate evidence-based recommendations. Managing Director of the Delft Digital Ethics Centre, Stefan Buijsman, noted that collaboration across institutions, disciplines, and governments enhances the global impact of such efforts.
Looking ahead, WHO is establishing a global consortium of collaborating centres on AI for health, bringing together leading institutions across its six regions to support countries in adopting AI responsibly. A preliminary meeting of prospective members was held from 17 to 19 March at TU Delft, where participants agreed on shared priorities and initial collaboration frameworks.
WHO said the initiative aims to build a coordinated global infrastructure to ensure that the governance of AI in health is guided by evidence, ethical standards, and the diverse needs of populations worldwide.





