Abstract / Summary
Summary Background The National Early Warning Score (NEWS) is widely used to predict patient deterioration, but retrospective validations face intervention bias and emphasise discrimination over clinical utility. We aimed to compare NEWS's clinical net benefit against simplified scoring rules and machine learning, to determine whether simpler approaches suffice or if complex models offer meaningful advantages, and to evaluate performance heterogeneity across patient subgroups. Methods We included fifteen Danish hospitals with over 2,08 million hospital encounters representing 825,200 unique patients over five years (2018 to 2023). We compared NEWS against both simpler and more complex approaches for predicting 24h mortality: Simplified NEWS (NEWS without blood pressure and temperature), DEWS (Simplified NEWS with age and sex), and a model based on eXtreme Gradient Boosting (XGB-EWS) incorporating vital signs, demographics, laboratory markers, plus medical history embeddings extracted using sentence transformers. We used propensity score weighting to mitigate intervention bias and evaluated performance using net benefit, Area Under the Receiver Operating Characteristic Curve (AUC), and calibration. Findings Decision curve analysis in the overall population showed maximum net benefit differences of 1,9 additional correct mortality identifications per 10000 patients between XGB-EWS and NEWS, and 1,5 per 10,000 between NEWS and Simplified NEWS. However, stratified analyses revealed substantial heterogeneity: in the high-risk group (initial NEWS >= 7), XGB-EWS provided a maximum gain of 78,1 correct identifications per 10,000 patients compared to NEWS, while Simplified NEWS resulted in a maximum performance loss of 60,2 per 10,000. Conversely, in low-risk groups, differences were minimal. Interpretation Model complexity produced limited average clinical utility gains, concentrated among patients initially classified as high risk by NEWS. These findings support prospective trials of risk-stratified monitoring based on the first NEWS assessment, testing simplified approaches for lower-risk patients while retaining full NEWS or electronic health-record models for higher-risk patients. Funding Novo Nordisk Foundation