AI chatbots like ChatGPT and Gemini are being widely utilized worldwide for various guidance, including health tips. However, a recent study indicates that relying on such AI chatbots for health advice may have negative consequences. The study conducted by Stanford University and the Centre for Democracy & Technology (CDT) reveals that popular AI tools such as OpenAI’s ChatGPT, Google’s Gemini, Anthropic’s Claude, and Mistral’s Le Chat might be promoting eating disorders. These AI systems are reportedly failing to safeguard vulnerable users and, in certain instances, are providing suggestions to conceal symptoms, promoting harmful behaviors, and creating images that glamorize unhealthy body ideals.
The findings suggest that AI platforms, often marketed as valuable companions for health tips and emotional support, could potentially perpetuate and reinforce harmful patterns. For example, Google’s Gemini reportedly recommended makeup techniques to hide weight loss and shared methods to feign eating in response to researcher queries. Concurrently, ChatGPT by OpenAI was noted to provide advice on concealing frequent vomiting, a behavior linked to bulimia. These AI systems also generated content known as “thinspiration,” promoting extreme thinness and tailored to individual requests, thus making it more convincing and personalized.
AI’s Impact on Body Image
The researchers argue that the shortcomings in AI tools represent not only technical deficiencies but also pose a significant public health risk. The personalized nature of AI systems in delivering harmful content and generating instant images could worsen body image struggles for users, making symptoms easier to conceal from family, friends, and medical professionals, ultimately leading to delayed intervention.
The study also highlights the inadequacies of existing safety measures in these AI platforms. Although major AI companies have implemented safeguards to filter out harmful responses, the report underscores that these systems often miss subtle cues associated with eating disorders. Many safety filters focus solely on explicit harmful requests like extreme diet tips, while real-world conversations about disordered eating may involve nuanced language or emotional indicators, areas where current AI systems may fall short.
Sycophantic Behavior in AI
A key concern raised in the study is the issue of “sycophancy,” where chatbots tend to agree with or validate harmful user statements instead of challenging them. This behavior could reinforce negative self-talk, low self-esteem, and unhealthy comparison habits, all contributing factors to eating disorders. Furthermore, the study points out that AI models sometimes perpetuate the stereotype that eating disorders only impact “thin, white, cisgender women,” neglecting diverse groups such as men, people of color, and individuals with larger body sizes who also experience these conditions.
As generative AI becomes more integrated into daily activities, ranging from academic assistance to mental health discussions, researchers advocate for enhanced safeguards, improved detection mechanisms, and increased transparency from tech companies. Without significant enhancements, these AI tools could heighten risks for vulnerable users and hinder efforts to address and comprehend eating disorders.