Loading your page…
Eurasian Journal of Emergency Medicine
By Bengisu Meral Ketenci and Demet Acar.
Ward brings these articles into the same app as your cards and questions. One free account for Ward, Suture, Osler and Prognia.
To evaluate the diagnostic and safety-related response performance of consumer multimodal large language model interfaces on a selected publication-derived challenge set of initially misdiagnosed dermatologic emergencies. Materials and Methods: This exploratory cross-sectional study evaluated 50 cases of initially misdiagnosed dermatologic emergencies, selected from published reports identified between May 1 and June 1, 2026. ChatGPT, Gemini, and Claude were tested on June 2, 2026, under text-only and text-plus-image conditions using a single standardized prompt. Outcomes were Top-1 and cumulative Top-4 accuracies, clinically appropriate emergency management, risk-triage failure, and harmful recommendations.
Top-1 accuracy ranged from 32.0% to 50.0% with text-only input and from 36.0% to 58.0% with text-plus-image input; corresponding Top-4 ranges were 50.0%-66.0% and 56.0%-74.0%. Appropriate management ranged from 38.0% to 54.0% and from 44.0% to 64.0%. No within-platform input-mode comparisons remained significant after Holm correction. Between-platform differences remained significant for Top-1 accuracy under both input conditions and for Top-4 accuracy and management appropriateness under text-plus-image input. Post-hoc differences primarily reflected lower observed performance for Claude than for Gemini in this dataset. Appropriate management occurred in 100% of Top-1-correct responses compared with 9.9% of Top-1-incorrect responses, whereas 18.0% of Top-4-correct responses had inappropriate management.
Within this selected challenge set and a single standardized prompting condition, the platforms demonstrated moderate diagnostic performance and variable performance on safety-related responses. Correct Top-1 diagnosis closely corresponded to appropriate management, whereas Top-4 recognition did not ensure safe management. The findings represent a single-run technical benchmark and do not establish output reproducibility, clinical usefulness, real-world safety, or stable platform superiority.
Published in Eurasian Journal of Emergency Medicine. The abstract is shown as published and remains the work of its authors; read the full article at the source.
eajem.com/articles/performance-of-consumer-multimodal-large-language-model-inter
Prognia lists this article for reference and education. It is not medical advice. Always read the original and follow current guidelines and local protocols before acting on it.
