Machine learning tools fueled by EHR data predicted high-risk surgical patients with greater accuracy than traditional methods.
Utilizing machine learning tools that leverage electronic health record data from a single organization could help providers predict patients at high risk of surgical complications more accurately than traditional approaches, a study published in PLOS Medicine found.
Complications arise in 15 percent of all US surgical procedures performed, the researchers noted, with high-risk surgeries resulting in complications up to 50 percent of the time. The expenses related to these events quickly add up: the total cost of surgical complications in the US is approximately $31.35 billion per year.
Organizations across the country have increased their efforts to identify high-risk patients and reduce surgical complications. However, if providers don’t have timely access to important patient data, or if organizations lack strong predictive models, it can be difficult to detect high-risk individuals.
Want to publish your own articles on DistilINFO Publications?
Send us an email, we will get in touch with you.
To improve the identification of high-risk patients, the researchers created a data repository that would store clinical and surgical EHR information from encounters at Duke University Health System.
The team then used the data repository to develop and train machine learning tools that would predict patients at high risk of postoperative complications. Predictor variables for the models included comorbidities, outpatient medications, and demographics such as age, sex, race, and smoking status.
The models achieved strong predictive performance, demonstrating areas under the curve ranging between 0.747 and 0.924.
The team also had human experts look at patients’ clinical factors, including multiple comorbidities and neurological disorders, and compared the experts’ performance to that of their machine learning models.
Researchers found that the machine learning models demonstrated a sensitivity of 77.24 percent, while the human experts achieved a sensitivity of 73.45 percent. The models also outperformed the experts in specificity, achieving a specificity of 74.92 percent compared to humans’ specificity of just 49.74 percent.
In terms of positive predictive value, the machine learning models also surpassed human experts, showing a PPV of 37.92 percent compared to the experts’ PPV of 22.47 percent.
In addition to outperforming human experts in predicting high-risk patients, the machine learning models surpassed the National Surgical Quality Improvement Program calculator, which is currently the most widely used pre-surgical risk prediction model.
The team found that their models achieved an AUC of 0.79, while the NSQIP calculator demonstrated an AUC of 0.67. The machine learning tools also achieved higher rates of sensitivity than the NSQIP calculator, at 0.9167 versus 0.7500, as well as higher specificity, at 0.5873 versus 0.5556.
The NSQIP calculator uses manually extracted EHR data, making it very difficult to update with new patient data or to adjust by adding new variables, the researchers stated. In contrast, the machine learning models developed in this study do not rely on manually extracted data. Instead, the models are built on real-time EHR data extraction, and can be continuously and automatically updated.
“Not only does this direct comparison between the two models provide further evidence that the NSQIP calculator does not perform strongly on our local patients, but it also puts forth a new methodology of local data extraction, curation, and modeling,” the researchers said.
“This new methodology is shown to be superior to ACS NSQIP’s for predicting postoperative complications in a local setting.”
The study did have its limitations, however, and the researchers stated that future research will need to add other data sources to further evaluate the models’ ability to predict high-risk patients. Still, the findings demonstrate that building machine learning models with EHR data from a single organization could help predict high-risk surgical patients.
“We demonstrated that machine learning models built from highly curated, clinically meaningful features from local, structured EHR data were able to achieve high sensitivity and specificity for classifying patients at risk of post-surgical complications,” the researchers concluded.
“Our models were shown to perform at a higher sensitivity and specificity through this analysis. By specifically targeting a narrower population of patients needing preoperative optimization, our healthcare system can better utilize clinical resources while lowering clinic costs.”
Date: December 19, 2018
Source: HealthIT Analytics
