The study involved challenging the free version of ChatGPT with real drug-related questions that were initially posed to Long Island University’s College of Pharmacy drug information service over a 16-month period in 2022 and 2023
Recent research presented at the American Society of Health-System Pharmacists Midyear Clinical Meeting has raised concerns about the reliability of ChatGPT, an artificial intelligence language model developed by OpenAI, in providing accurate and complete drug-related information. The study, led by Sara Grossman, PharmD, Associate Professor of Pharmacy Practice at Long Island University, revealed that nearly three-quarters of the responses generated by ChatGPT were deemed incomplete or incorrect, posing potential risks to patients.
The study involved challenging the free version of ChatGPT with real drug-related questions that were initially posed to Long Island University’s College of Pharmacy drug information service over a 16-month period in 2022 and 2023. Pharmacists from the university first researched and answered 45 questions, establishing a baseline for comparison. After excluding six questions due to a lack of literature, 39 questions were used to assess ChatGPT’s responses.
Key Findings
Accuracy and Completeness
Out of the 39 questions posed to ChatGPT, only 10 responses were deemed satisfactory according to the study’s criteria. The remaining 29 responses either did not directly address the question, were inaccurate, or were incomplete.
Reference Citations
Researchers requested references from ChatGPT to verify the information provided. However, references were only provided in eight responses, and each included non-existent citations, raising concerns about the reliability of the sources used by the AI model.
Example of Inaccuracy
In one instance, ChatGPT was asked about a potential drug interaction between the COVID-19 antiviral Paxlovid and the blood-pressure lowering medication verapamil. The AI indicated no reported interactions, contrary to the reality that these medications have the potential to interact, leading to excessive lowering of blood pressure.
Implications and Caution
Sara Grossman emphasised the need for caution among healthcare professionals and patients when using ChatGPT as an authoritative source for medication-related information. She urged verification of information through trusted sources to ensure patient safety.
Gina Luchen, PharmD, ASHP director of digital health and data, highlighted the potential impact of AI-based tools on clinical and operational aspects of care. Pharmacists were advised to be vigilant stewards of patient safety, evaluating the appropriateness and validity of specific AI tools for medication-related uses.
As artificial intelligence continues to play a role in healthcare, this study underscores the importance of thoroughly evaluating the accuracy and reliability of AI models, especially when it comes to providing critical drug-related information. The findings emphasise the need for ongoing education on trusted sources for medication information and a cautious approach to the integration of AI tools in the healthcare system. Tina Zerilli, PharmD, will present further insights into the evaluation of ChatGPT’s ability to respond to drug information questions, shedding light on the challenges and opportunities in utilising AI for medication-related queries.