Development and Temporal Validation of a Logistic Regression–Based Risk Score for Identifying Life-Threatening Chest Pain at Emergency Department Triage: A Retrospective Cohort Study

Dongsheng Xu,1 Ping Gao,1 Qiang Xu,2 Huanhuan Nie31Department of Emergency Medicine, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, People’s Republic of China; 2Department of Neurology, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, People’s Republic of China; 3Department of Otolaryngology-Head and Neck Surgery, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, People’s Republic of ChinaCorrespondence: Huanhuan Nie, Department of Otolaryngology-Head and Neck Surgery, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, People’s Republic of China, Email 531583550@qq.comBackground: Chest pain is a frequent emergency department (ED) presentation with highly time-dependent outcomes. Widely used risk tools such as HEART and GRACE incorporate laboratory results that are unavailable at first nurse contact, limiting their value for immediate triage. In mainland China, acuity assignment follows the Chinese Emergency Triage Scale (CETS), yet nurse triage accuracy remains variable, particularly among less experienced staff.Objective: To develop and temporally validate a simplified, non-laboratory clinical risk score that identifies life-threatening conditions (LTCs)—defined as final physician-assigned CETS Level I or II—among adults presenting to the ED with chest pain, and to map score thresholds onto the four-level CETS framework for nurse-facing decision support.Methods: This was a single-centre retrospective observational cohort study, reported in accordance with TRIPOD+AI, at an ED of a large tertiary academic hospital in Hefei, Anhui Province, China. The cohort was split by calendar time into an internal development cohort (n = 2,005; July 2020–February 2021) and a temporal validation cohort (n = 501; February–March 2021). The internal cohort was randomly divided into a training set (n = 1,391) and a test set (n = 614). Candidate predictors were restricted to information routinely available at nurse triage. Least absolute shrinkage and selection operator (LASSO) regression identified parsimonious predictors, which were entered into a multivariable logistic regression (LR) model. Six machine-learning (ML) algorithms were trained as benchmarks. Discrimination (AUC, AUPRC), calibration (calibration curves, Brier score), clinical utility (decision-curve analysis, standardised net benefit), and DeLong tests for AUC differences were evaluated. An integer risk score was derived from LR coefficients. Feature contributions were examined with SHAP values based on the LR model.Results: LTC prevalence was 44.8% (899/2,005) in the internal cohort and ED mortality was 1.7% (34/2,005). LASSO retained 13 predictors: age, sex, mode of arrival, symptom onset time, cardiogenic chest pain (CCP), chest tightness, hypertension, diabetes, coronary artery disease (CAD), chronic kidney disease (CKD), cerebrovascular disease (CVD), shock index (SI), and Modified Early Warning Score (MEWS). The LR model showed stable discrimination—AUC 0.864 (95% CI 0.844– 0.883) in training, 0.876 (0.849– 0.904) in the test set, and 0.860 (0.825– 0.896) in temporal validation—without statistically significant AUC superiority of more complex ML models on the test or validation sets (DeLong P > 0.05). In the test set, LR achieved the highest AUPRC (0.858), F1 score (0.792), and standardised net benefit (0.690), with the lowest Brier score (0.142). The integer score ranged from − 3 to 49. At the Youden-optimal cut-off of 18.8, four CETS-aligned tiers were defined: Level IV (score < 6.4), Level III (6.4– 18.8), Level II (18.8– 32.1), and Level I (score > 32.1). SHAP analysis of the LR model ranked transportation, CCP, MEWS, SI, sex, and age among the strongest contributors.Conclusion: An internally and temporally validated, laboratory-free LR-based integer score showed discrimination and calibration comparable to more complex ML models, with clearer interpretability for ED nursing practice. The score is intended to support—not replace—clinical judgement. Multicentre external validation and prospective implementation studies are required before routine clinical adoption.Keywords: chest pain, clinical prediction model, emergency department, life-threatening condition, logistic regression, risk assessment, triage, nursing