As many people face gaps in mental healthcare access, they increasingly rely on artificial intelligence (AI) chatbots for help with side effects from psychiatric medications. However, a new study from the Georgia Institute of Technology shows these AI tools often fall short in detecting and responding to these complex and risky situations.
The research evaluated how well large language models (LLMs), including popular AI chatbots, identify adverse drug reactions and offer useful advice. While the AI could mimic the tone of a psychiatrist, it often failed to accurately recognize medication side effects or provide clear, actionable guidance.
Key Findings:
- AI chatbots frequently misidentify psychiatric medication side effects or give vague advice that users cannot act on.
- Although the AI mimics a compassionate and professional tone, its clinical recommendations do not meet expert standards.
- This raises concerns about the risks of relying on AI chatbots for urgent mental health issues, especially among underserved populations with limited access to real healthcare providers.
AI in Mental Health: A Double-Edged Sword
AI chatbots powered by large language models are available 24/7 and free to use, making them attractive resources. Many people with mental health conditions now turn to these tools for quick advice when facing medication side effects — a situation far more complex than typical information queries.
“This is especially important for communities that lack access to mental healthcare,” said Mohit Chandra, a Ph.D. student and lead author of the study. “AI tools could be life-changing if they were safer and more reliable.”
The Study
Led by Professor Munmun De Choudhury and Chandra at Georgia Tech, the team developed a new method to test how well AI chatbots can detect and respond to adverse drug reactions. They collaborated with psychiatrists to create a clinical standard and compared it against AI responses.
The researchers gathered real-world data from Reddit, where many users discuss medication side effects. They tested nine different LLMs, including general models like GPT-4o and LLaMA-3.1, as well as specialized medical AI.
The evaluation looked at how accurately the AI identified side effects and categorized their types. They also assessed the AI’s emotional tone, readability, harm-reduction strategies, and whether the advice was actionable.
Results Highlight AI Limitations
While AI chatbots sounded empathetic and professional, they struggled to understand the subtle details of adverse reactions. They often failed to distinguish between different side effects and rarely offered practical, expert-level advice.
Implications and Next Steps
The researchers hope their findings will guide developers to build safer and more effective AI tools tailored to mental health needs. Chandra emphasized the importance of improving AI for those with limited healthcare access.
“AI is always available and can communicate in many languages, which is a huge benefit,” Chandra explained. “But if it gives wrong information, the consequences can be serious.”
The study underlines the urgent need to refine AI chatbots so they can provide accurate, personalized, and actionable support for mental health users.
Related Topics
