Abstract / Summary
Abstract This study aimed to construct and validate a machine learning-based model for early prediction of delirium risk in patients with severe pneumonia. Data on patients with severe pneumonia were extracted from the Medical Intensive Care Information Database IV (MIMIC-IV) and divided into a training set and an internal validation set at a ratio of 7:3. External validation was conducted using prospective data from a single center in southwest China. Potential feature variables were screened using Least Absolute Shrinkage and Selection Operator (LASSO), Random Forest(RF), and Adaptive Boosting (AdaBoost), and eight machine learning algorithms were employed to construct early prediction models for delirium in critically ill pneumonia patients, with the optimal algorithm model selected. Model performance was evaluated using multiple metrics, including the area under the curve (AUC), brier score, and decision curve analysis (DCA). The importance of predictive factors was ranked using the Shapley Additive Explanations (SHAP), and the streamlit framework was used to build an interactive web-based calculator. Among 8,059 critically ill patients in the MIMIC database, the incidence of delirium was 71.36%, with 5,641 cases assigned to the training set and 2,418 cases included in the internal validation set. External validation was conducted on 133 critically ill pneumonia patients in southwestern China, with a delirium incidence rate of 55.6%. In the internal validation set, the AUC for the eight machine learning models ranged from 0.58 to 0.73. Among them, the Light Gradient Boosting Machine (LightGBM) model performed the best, with an accuracy of 0.75, precision of 0.77, recall of 0.75, positive predictive value(PPV) of 0.77, negative predictive value(NPV) of 0.63, F1 score of 0.72, and Brier score of 0.175. The LightGBM model achieved an AUC of 0.70 and a Brier score of 0.223 in the external validation set. Further DCA demonstrated that when the threshold probability ranged from 0 to 0.68, the net clinical benefit of the LightGBM model was consistently higher than that of the Treat-All and Treat-None strategies. SHAP algorithm feature importance analysis indicated that the three clinical features most influential on the LightGBM model output were sedatives_used, apsiii, and ventilation_status. This study demonstrates that machine learning-based models have the potential to be used for the early identification of high-risk patients with delirium in severe pneumonia. The interactive web-based calculator may serve as a practical predictive management tool for delirium in patients with severe pneumonia.