A recent study published in JAMA Internal Medicine has provided an early glimpse into the potential role that artificial intelligence (AI) assistants like ChatGPT could play in medicine. Led by Dr. John W. Ayers from the Qualcomm Institute at the University of California San Diego, the study compared written responses from physicians and those from ChatGPT to real-world health questions. A panel of licensed healthcare professionals preferred ChatGPT’s responses 79% of the time, rating them as higher quality and more empathetic.
Image Credit: PopTika via Shutterstock / HDR tune by Universal-Sci
Assessing ChatGPT’s Role in Healthcare
The research team aimed to determine whether ChatGPT could accurately respond to questions patients send to their doctors. If so, AI models could be integrated into health systems to improve physician responses to patient questions and alleviate the growing burden on physicians.
“The COVID-19 pandemic accelerated virtual healthcare adoption,” said study co-author Dr. Eric Leas. “While this made accessing care easier for patients, physicians are burdened by a barrage of electronic patient messages seeking medical advice that have contributed to record-breaking levels of physician burnout.”
Study Design and Results
The team randomly sampled 195 exchanges from Reddit’s AskDocs subreddit, where a verified physician responded to a public question. They provided the original question to ChatGPT and asked it to author a response. A panel of three licensed healthcare professionals assessed each question and the corresponding responses, blinded to whether the response originated from a physician or ChatGPT.
The panel of healthcare professional evaluators preferred ChatGPT responses to physician responses 79% of the time. ChatGPT responses were rated significantly higher in quality and empathy than physician responses.
Harnessing AI Assistants for Patient Messages
The study suggests that tools like ChatGPT can efficiently draft high-quality, personalized medical advice for review by clinicians. “Instead, a physician harnessing ChatGPT is the answer for better and empathetic care,” said Dr. Adam Poliak, an assistant professor of Computer Science at Bryn Mawr College and study co-author.
Dr. Christopher Longhurst, Chief Medical Officer and Chief Digital Officer at UC San Diego Health, added, “Our study is among the first to show how AI assistants can potentially solve real-world healthcare delivery problems.” UCSD Health has already begun integrating ChatGPT into their processes.
Investments into AI assistant messaging could impact patient health and physician performance, with potential applications in training doctors in patient-centered communication, eliminating health disparities, building new medical safety systems, and delivering higher quality and more efficient care.
If you are interested in more details about the underlying study, be sure to check out the paper published in the peer-reviewed science journal JAMA Internal Medicine listed below.
Sources and further reading:
Too busy to follow science news during the week? - Consider subscribing to our (free) newsletter - (Universal-Sci Weekly) - and get the 5 most interesting science articles of the week in your inbox
FEATURED ARTICLES:
