AI can help improve emergency room admission decisions, research shows

Generative artificial intelligence (AI), such as GPT-4, can help predict whether a patient needs to be admitted to the emergency room, even with minimal training on a limited number of records, according to researchers at the Icahn School of Medicine. Mount Sinai.

Details of the study were published in the May 21 online issue of the Journal of the American Medical Informatics Association in an article titled “Evaluating the Accuracy of a State-of-the-Art Large Language Model for Predicting Emergency Room Admissions.”

In the retrospective study, researchers analyzed data from seven hospitals in the Mount Sinai Health System, using both structured data, such as vital signs, and unstructured data, such as nurse triage notes, from more than 864,000 emergency department visits, excluding identifiable patient data. Of these visits, 159,857 (18.5%) resulted in patient admission to the hospital.

The researchers compared GPT-4 with traditional machine learning models such as Bio-clinical-BERT for text and XGBoost for structured data in different scenarios, assessing its performance to predict hospital admissions independently and in combination with the traditional methods.

“We were motivated by the need to test whether generative AI, specifically large language models (LLMs) like GPT-4, could improve our ability to predict admissions in high-volume environments such as the emergency department,” says co -senior author Eyal. Klang, MD, director of the Generative AI Research Program at the Department of Data-Driven and Digital Medicine (D3M) at Icahn Mount Sinai.

“Our goal is to improve clinical decision making through this technology. We were surprised by how well GPT-4 adapted to the ED setting and provided the reasoning for its decisions. This ability to explain its rationale sets it apart of traditional models and opens new avenues for AI in medical decision-making.”

While traditional machine learning models use millions of records for training, LLMs can effectively learn from just a few examples. Additionally, LLMs can integrate traditional machine learning predictions, improving performance, according to the researchers

“Our research suggests that AI could soon support emergency room physicians by making quick, informed decisions about patient admissions. This work opens the door for further innovation in healthcare AI, driving the development of models that can reason and learn from limited data the way human experts do,” said co-senior author Girish N. Nadkarni, MD, MPH, Irene and Dr. Arthur M. Fishberg Professor of Medicine at Icahn Mount Sinai, Director of the Charles Bronfman Institute of Personalized Medicine, and System Chief of D3M.

“While the results are encouraging, the technology still plays a supporting role, improving the decision-making process by providing additional insights, rather than taking over the human component of healthcare, which remains critical.”

The research team is investigating how large language models can be applied to healthcare systems, with the aim of harmoniously integrating them with traditional machine learning methods to address complex challenges and decision-making in real-time clinical environments.

“Our research shows how LLMs can be integrated into healthcare operations. The ability to quickly train LLMs highlights their potential to provide valuable insights, even in complex environments such as healthcare,” said Brendan Carr, MD, MA, MS, a study co author and emergency room physician who is Chief Executive Officer of Mount Sinai Health System.

“Our study sets the stage for further research into AI integration in healthcare across the many domains of diagnostic, treatment, operational and administrative tasks that require continuous optimization.”