ChatGTP outperforms physicians in providing high-quality advice to patients
University of California San Diego study compared written responses from physicians and those from ChatGPT to real-world health questions
[Apr. 30, 2023: Staff Writer, The Brighter Side of News]
The study compared written responses from physicians and those from ChatGPT to real-world health questions. (CREDIT:Creative Commons)
A new study led by Dr. John W. Ayers from the Qualcomm Institute at the University of California San Diego shows how artificial intelligence (AI) assistants, like ChatGPT, could play a role in healthcare.
The study compared written responses from physicians and those from ChatGPT to real-world health questions, and a panel of licensed healthcare professionals preferred ChatGPT’s responses 79% of the time, rating them higher quality and more empathetic.
The research team set out to answer the question of whether ChatGPT could respond accurately to questions patients send to their doctors. If yes, AI models could be integrated into health systems to improve physician responses to questions sent by patients and ease the ever-increasing burden on physicians. The study, published in JAMA Internal Medicine, has important implications for the future of healthcare.
Background
The use of AI in healthcare is not a new concept, but it has yet to be fully realized. The potential applications of AI in healthcare are vast, and its use could revolutionize the industry. AI could help improve patient outcomes, reduce healthcare costs, and increase efficiency.
Related Stories
One area where AI could have a significant impact is in physician-patient interactions. Physicians are often overwhelmed with the number of messages they receive from patients seeking medical advice. AI assistants could be used to help physicians respond to these messages more efficiently, freeing up time for more critical tasks.
AI assistants, like ChatGPT, are a type of AI model that uses natural language processing to understand and generate human-like responses to written or spoken input. ChatGPT is an advanced AI model based on the GPT-3.5 architecture developed by OpenAI. It is capable of generating responses that are almost indistinguishable from those written by a human.
The Study
To determine the efficacy of ChatGPT in responding to patient messages, the research team turned to social media. They used r/AskDocs, a subreddit with approximately 452,000 members, where patients post medical questions and verified healthcare professionals submit answers.
r/AskDocs, a subreddit with approximately 452,000 members
The team randomly sampled 195 exchanges from AskDocs where a verified physician responded to a public question. The team provided the original question to ChatGPT and asked it to author a response. A panel of three licensed healthcare professionals assessed each question and the corresponding responses and were blinded to whether the response originated from a physician or ChatGPT. They compared responses based on information quality and empathy, noting which one they preferred.
The panel of healthcare professional evaluators preferred ChatGPT responses to physician responses 79% of the time. ChatGPT responses were rated significantly higher in quality than physician responses, and they were also more empathic. Empathetic or very empathetic responses were 9.8 times higher for ChatGPT than for physicians (physicians 4.6% versus ChatGPT 45.1%).
Example Questions with Physician and Chatbot Responses. (CREDIT: JAMA)
The study's authors note that while the study pitted ChatGPT against physicians, the ultimate solution isn't throwing out physicians altogether. Instead, a physician harnessing ChatGPT is the answer for better and empathetic care.
The use of AI assistants in healthcare messaging could improve workflow, patient health, and physician performance. The authors suggest that integrating AI assistants into healthcare messaging be done in the context of a randomized controlled trial to judge how the use of AI assistants impact outcomes for both physicians and patients.
Implications for Healthcare
The study has important implications for the future of healthcare. The COVID-19 pandemic has accelerated the adoption of virtual healthcare, making accessing care easier for patients. However, physicians are burdened by a barrage of electronic patient messages seeking medical advice that have contributed to record-breaking levels of physician burnout. AI assistants could help alleviate this burden, freeing up physicians to focus on more critical tasks.
Kernel density plots are shown for the average across 3 independent licensed health care professional evaluators using principles of crowd evaluation. A, The overall quality metric is shown. B, The overall empathy metric is shown. (CREDIT: JAMA)
“The opportunities for improving healthcare using AI are vast,” said Dr. Ayers. “AI models like ChatGPT could be integrated into health systems to improve physician responses to patient questions, freeing up their time and reducing physician burnout. Additionally, AI could help with early diagnosis and treatment, reducing the overall cost of healthcare and improving patient outcomes. However, it is important to note that AI is not a replacement for physicians, but a tool to assist them in providing better, more empathetic care.”
As the healthcare industry continues to evolve, the use of AI in patient care will undoubtedly become more prevalent. AI assistants like ChatGPT offer a promising solution to the problem of physician burnout, and their use could improve patient outcomes and reduce healthcare costs. With further research and development, AI could transform healthcare for the better, making it more accessible, affordable, and effective for patients and providers alike.
Note: Materials provided above by The Brighter Side of News. Content may be edited for style and length.
Like these kind of feel good stories? Get the Brighter Side of News' newsletter.
