Loading your page…
BMC Emergency Medicine
By Ji Han Heo, Gil Joon Suh, Han‐Yeong Jeong and 1 others.
Ward brings these articles into the same app as your cards and questions. One free account for Ward, Suture, Osler and Prognia.
Code stroke activations are widely implemented to accelerate diagnostic evaluation of suspected acute stroke in the emergency department. While effective in expediting care, false positive activations disrupt workflows, consume resources, and delay evaluation for other patients. We evaluated whether adaptations of a large language model could diagnostically confirm suspected acute stroke using information from initial emergency department documentation and compared results to BioClinical BERT. We analyzed initial ED clinical notes from Seoul National University Hospital (January 2018 to December 2022). Notes contained code-mixed (Korean and English), semi-structured and unstructured free text describing presenting illness, past medical history, and neurological deficits of patients presenting with acute neurological symptoms within 24 h of onset. LLaMA-3.1-70B-Instruct-GPTQ-INT4 was used to translate each text to English and augment negative labels in training folds. LLaMA-3.1-8B-Instruct was fine-tuned using (1) quantized low-rank adaptation and (2) a self-refinement instruction-tuning strategy applied with clinical rule of thumb augmentation. Performance was evaluated on the validation folds for AUROC, AUPRC, F1, accuracy, precision, recall, specificity, and Brier Score. Based on per-seed aggregation followed by average across seeds, BioClinical BERT yielded AUROC of 0.7230 ± 0.0200 (95% CI, 0.7011–0.7449) and Brier Score of 0.2081 ± 0.0049. Self-refinement, instruction-tuning of quantized low-rank adaptation with clinical rule of thumb augmentation yielded AUROC of 0.7882 ± 0.0102 (95% CI, 0.7770–0.7994) and Brier Score of 0.1577 ± 0.0020. A paired bootstrap comparison of AUROC with per-seed resampling (2000 iterations) and Fisher’s method for p-value aggregation demonstrated a statistically significant improvement ( p < 0.001). Self-refinement instruction-tuning combined with quantized low-rank adaptation and clinical rule of thumb guidance outperformed BioClinical BERT and offers a scalable approach for automated diagnostic confirmation of suspected acute stroke using initial emergency department documentation. External validation is still required prior to deployment.
Published in BMC Emergency Medicine. The abstract is shown as published and remains the work of its authors; read the full article at the source.
Prognia lists this article for reference and education. It is not medical advice. Always read the original and follow current guidelines and local protocols before acting on it.
