Frontiers in Artificial Intelligence
Performance evaluation of ARIMA, VAR, ANN, GRU, and LSTM for multivariate healthcare time-series prediction: a rigorous, reproducible benchmarking study with ablation analysis on MIMIC-III and eICU
By Md. Tazul Islam, Muhammad Hanif and Samuel Kim.
Abstract
Introduction Accurate forecasting of physiological time-series in the intensive care unit (ICU) is critical for early-warning systems, deterioration detection, and dynamic resource allocation. Yet the comparative literature suffers from five persistent methodological deficiencies: non-unified experimental conditions, heterogeneous preprocessing, univariate classical baselines compared against multivariate neural models, single-run point estimates without variance, and underspecified prediction tasks. Methods This paper addresses all five deficiencies through a standardized, reproducible benchmarking study comparing Autoregressive Integrated Moving Average (ARIMA), Vector Autoregression (VAR), Artificial Neural Networks (ANN), Gated Recurrent Units (GRU), and Long Short-Term Memory (LSTM) networks on one-step-ahead (1-h-ahead) prediction of heart rate, respiratory rate, systolic and diastolic blood pressure, and oxygen saturation across two large-scale ICU cohorts (8,000 MIMIC-III and 15,000 eICU admissions). VAR is introduced as a multivariate linear baseline alongside univariate ARIMA, enabling a three-way decomposition of performance gains into multivariate modeling, nonlinear approximation, and gated temporal memory. All experiments are repeated over 10 independent seeds and reported as mean ± 95% confidence interval with Diebold-Mariano, paired t -test, and Bonferroni-corrected ANOVA significance testing. Seven categories of ablation study, a clinical alarm simulation, a leave-one-hospital-out protocol, and a subgroup fairness analysis are additionally conducted. Results LSTM achieves the highest performance on MIMIC-III (RMSE 0.082 ± 0.004, R 2 0.940 ± 0.008, directional accuracy 90.1%±0.9%), with GRU reaching near-equivalent accuracy at 64% of training cost. The layered decomposition reveals that gated temporal memory provides the largest marginal gain (−47% to −53% RMSE over ANN), followed by nonlinear approximation (−19% over VAR) and multivariate modeling (−13% over ARIMA). Ablation identifies normalization as the single most critical preprocessing step (+79.3% RMSE without it) and a window length of l = 10 as optimal. The clinical alarm simulation demonstrates a 38.4% false alarm rate reduction for LSTM relative to threshold-only monitoring, and the leave-one-hospital-out experiment confirms cross-institutional generalizability with consistent model ranking. Discussion Subgroup fairness analysis identifies a performance gap for severely ill patients (Sequential Organ Failure Assessment score ≥ 8, ratio 1.27) warranting targeted future work. These results establish a reusable, methodologically rigorous benchmark for ICU physiological forecasting and provide actionable model-selection guidance for clinical deployment.







