Hospital-acquired acute kidney injury (HA-AKI) poses risks like chronic kidney disease, longer hospital stays, and higher costs. Mass General Brigham Digital researchers assessed the Epic Risk of HA-AKI predictive model, revealing moderate success in predicting HA-AKI risk. However, performance was lower than Epic Systems Corporation’s internal validation, emphasizing the need for rigorous AI model validation before clinical use.
After training the model using data from MGB hospitals, the researchers tested it on data from nearly 40,000 inpatient hospital stays for a five-month period between August 2022 and January 2023. The dataset was extensive with many points collected on patient encounters, including information such as patient demographics, comorbidities, principal diagnoses, serum creatinine levels and length of hospital stay. Two analyses were completed looking at encounter-level and prediction-level model performance.
AI Tool’s Reliability in Predicting Hospital-Acquired Kidney Injury Risk
The investigators observed that the tool was more reliable when assessing patients with lower risk of HA-AKI. Although the model could confidently identify which low-risk patients would not develop HA-AKI, it struggled to predict when higher-risk patients might develop HA-AKI. Results also varied depending on the stage of HA-AKI being evaluated —predictions were more successful for Stage 1 HA-AKI compared to more severe cases.
The authors concluded overall that implementation may result in high false-positive rates and called for further study of the tool’s clinical impact.
“We found that the Epic predictive model was better at ruling out low-risk patients than identifying high-risk patients,” said lead study author Sayon Dutta, MD, MPH, of Mass General Brigham Digital’s Clinical Informatics team, and an emergency medicine physician at Massachusetts General Hospital. “Identifying HA-AKI risk with predictive models could help support clinical decisions such as by warning providers against ordering nephrotoxic medications, but further study is needed before clinical implementation.”
Source-Eurekalert
