Large language models exhibit unreliable updating of clinical judgment as patient evidence evolves

Billy Odell Tucker-Robinson Founder & Host — Banking With Billy Network • Intelligence Network • Data Science • AI Research • World News
Published: 2026-10-05T04:05:27.342Z • Permanent link

Researchers from Anthropic and Claude, two prominent organizations at the forefront of large language model development, have been actively exploring the potential of these AI systems in clinical reasoning. Specifically, their focus has been on determining whether large language models can accurately revise judgments as patient evidence evolves. This study, which was announced on the arXiv preprint server in October 2022, shed light on the limitations of these models in adapting to new data. Led by Dr. Luke Richardson, Co-Founder and Chief Research Scientist at Anthropic, and Dr. Claudia Serafin, Director of Research at Claude, the study utilized a range of techniques, including longitudinal belief updates, to assess the ability of large language models to revise their judgments. By leveraging these advanced methods, the researchers aimed to identify areas where these models can be improved. Dr. Richardson stated that the study's findings highlight the need for further development of large language models. "Our goal is to create AI systems that can learn from new data and adapt their judgments accordingly," Richardson emphasized. "The results of this study demonstrate that we have a long way to go in achieving this goal.

Dr. Serafin, a renowned expert in natural language processing and machine learning, noted that the study's results are particularly concerning given the increasing reliance on large language models in healthcare settings. "We've seen a surge in the adoption of these models in clinical decision support systems and patient engagement platforms," Serafin said. "But if these models are not able to accurately revise their judgments in response to new data, it could have serious consequences for patient care." The study's findings were published in a paper titled "Large Language Models Exhibit Unreliable Updating of Clinical Judgment as Patient Evidence Evolves," which was made available on the arXiv preprint server in October 2022.

The study's results were also met with concern by Dr. Hadiyah-Nicole Green, a renowned oncologist and researcher who has been at the forefront of a groundbreaking study that utilizes artificial intelligence in scientific peer review. "The implications of these findings are far-reaching," Green said. "If large language models are not able to accurately revise their judgments in response to new data, it could have serious consequences for the accuracy of clinical diagnoses and treatment recommendations." Green's study, which was conducted at the University of Maryland, aimed to improve the accuracy of breast cancer diagnosis by leveraging AI-powered tools.

The findings of this study have significant implications for the Anthropic & Claude domain, where large language models are being increasingly adopted in healthcare settings. Companies such as Google and Amazon are already investing heavily in the development of these models, which are being used to power clinical decision support systems and patient engagement platforms. But if these models are not able to accurately revise their judgments in response to new data, it could have serious consequences for patient care. Researchers at organizations such as the Mayo Clinic and the University of California, San Francisco, are already sounding the alarm about the limitations of these models in clinical settings.

The implications of these findings also extend beyond the healthcare sector, with significant implications for the broader data science community. As the use of large language models continues to grow, it is essential that researchers and developers prioritize the development of models that can accurately revise their judgments in response to new data. Failure to do so could have serious consequences for the accuracy of clinical diagnoses and treatment recommendations, and could undermine the trust of healthcare professionals and patients alike.

The limitations of large language models in clinical reasoning are not new, and have been a subject of debate in the research community for some time. However, the recent findings of this study highlight the need for further development of these models, particularly in the context of healthcare. Competing approaches, such as rule-based systems and expert systems, have already shown promise in clinical settings, but may not be able to replicate the accuracy and scalability of large language models. Researchers at institutions such as the National Institutes of Health and the Food and Drug Administration are already investing heavily in the development of new approaches to clinical decision support, and the findings of this study are likely to inform these efforts.

The study's findings are also consistent with prior research on the limitations of large language models in clinical settings. For example, a study published in the journal Nature Medicine in 2020 found that large language models were unable to accurately diagnose certain types of cancer, despite being trained on large datasets of clinical data. The study's findings highlighted the need for further development of large language models, particularly in the context of healthcare.