ChatGPT outperforms clinicians in post-negative test disease probability

ChatGPT-4 outperformed human clinicians in determining pretest and post-test disease probability after a negative test result involving chest radiographs and mammograms, according to a research letter published December 11 in JAMA Network Open. Investigators led by Adam Rodman, MD, from Beth Israel Deaconess Medical Center in Boston did find, however, that ChatGPT-4 did not perform as well after positive test results. “However, even if imperfect, probabilistic recommendations from [large language models] might improve human diagnostic performance through collective intelligence, especially if AI diagnostic aids can combine probabilistic, narrative, and heuristic approaches to diagnosis,” Rodman and colleagues wrote. Imaging tests are a first-line tool for determining diagnoses, but the researchers underscored that health practitioners “often perform poorly” at estimating probabilities of disease before and after imaging exams are performed. Medical researchers over the past year have experimented with using large language models to help with clinical workflows and assist with disease diagnosis, and previous reports suggest these models can understand clinical reasoning to an extent. Rodman and co-authors explored the ability of one such model, ChatGPT-4, to perform probabilistic reasoning. They compared its performance with a survey of 553 human clinicians from various specialties. The clinicians performed probabilistic reasoning in a series of five cases with s...
Source: AuntMinnie.com Headlines - Category: Radiology Authors: Tags: Advanced Visualization Source Type: news