Stanford Center for Artificial Intelligence in Medicine and Imaging (AIMI) reposted this
The result from our study, that performance of LLM + physicians = LLM alone on the set of medical challenges considered, indicates that we were not harnessing LLMs (in the context of the scenarios evaluated) to ideally complement human intellect, per setup, training, context. Much to do and explore on better training of physicians to use LLMs and, more fundamentally, on methods for advancing human-AI collaboration, in pursuit of dream of leveraging AI to augment human cognition. Stanford Institute for Human-Centered Artificial Intelligence (HAI) Stanford Center for Artificial Intelligence in Medicine and Imaging (AIMI)
AI & MEDICAL DIAGNOSES - A new study from Stanford Biomedical Data Science Program has evaluated the impact of GPT-4 on physicians' diagnostic reasoning compared to conventional diagnostic resources: - 50 physicians (attending and resident) participated in a clinical vignette study comparing GPT-4 and traditional diagnostic tools. - Physicians using GPT-4 achieved a median diagnostic score of 76.3%, compared to 73.7% for those using conventional resources—a modest 1.6 percentage point difference. - GPT-4 alone outperformed both groups by 15.5 percentage points in diagnostic reasoning accuracy. - Physicians using GPT-4 spent an average of 82 seconds less on cases compared to the conventional group. - Results suggest opportunities to improve physician-AI collaboration in clinical practice, especially in enhancing diagnostic efficiency. "GPT-4 demonstrated higher performance than both physician groups, suggesting opportunities for further improvement in physician-AI collaboration." — Stanford University Research Team Click below to read the full study 👇that included contributions from Hannah Kerman, Josephine Cool, Zahir Kanjee, Eric Horvitz, Arnold Milstein, Jonathan H. Chen, Adam Rodman, Jason Hom, Neera Ahuja, Yingjie(Isabel) Weng and others. Opinions? Join the conversation on Ethan Mollick's post: https://lnkd.in/dTp63Q_b Ask AI is an independent nonprofit that's been helping people stay one step ahead of artificial intelligence opportunities and disruptions since 2017: Newsletter: https://lnkd.in/gA2aB_Ef Upload demo: https://lnkd.in/gc43EUDu Guest post: https://lnkd.in/gV8wVAVx Fundraising? https://lnkd.in/ga7q5Ned #research #artificialintelligence #AIinHealthcare #medicaldiagnosis #gpt4 #doctors #medicalprofession