An Exploratory Study: Can Vision-Enabled Large Language Models Interpret Selected Static Urologic Images?

IntroductionVision-enabled large language models (LLMs) are increasingly capable of interpreting medical images.However, performance across common urologic imaging modalities remains poorly defined.We compared four publicly available vision-enabled LLMs on their ability to identify imaging modalities and interpret selected static urologic images at a single point in time. MethodsThis exploratory observational study evaluated 16 selected static urologic images comprising radiography, US, computed tomography (CT), and MRI (four images per modality).Four publicly available vision-enabled LLMs (ChatGPT-4o (OpenAI, Inc., San Francisco, CA, USA), Grok 2.0 beta (SpaceXAI LLC, San Francisco, CA, USA), Gemini 1.5 (Google DeepMind, London, UK), and Copilot Azure AI Vision 3.2 (Microsoft Corporation, Redmond, WA, USA)) identified the imaging modality and interpreted each image before and after the addition of an arrow highlighting the principal abnormality.Each model was accessed through its public consumer interface between May 1 and May 10, 2025, and generated a single response per image per condition.Responses were graded by one unblinded investigator using a predefined 5-point Likert scale based on interpretation accuracy. ResultsChatGPT and Grok correctly identified the imaging modality in all 16 images (100%).Copilot correctly identified 10 of 16 images (62.5%), misclassified two (12.5%), and declined to interpret four (25.0%; three US and one MRI) because of content-filtering restrictions.Gemini declined to interpret all images.Among the 12 images interpreted by all three models, median interpretation scores without arrow context were 4.0 (IQR 3.0-5.0)for ChatGPT, 4.0 (IQR 3.0-4.3)for Grok, and 2.0 (IQR 2.0-4.0) for Copilot (Friedman P = 0.004).Addition of contextual arrows was not associated with a statistically detectable change in interpretation accuracy for any model (all P ≥ 0.20), although the CIs were wide and a clinically meaningful effect cannot be excluded. ConclusionsPerformance varied substantially among publicly available vision-enabled LLMs.ChatGPT and Grok achieved higher interpretation scores than Copilot on this limited exploratory image set, whereas Copilot and Gemini were constrained by content moderation policies.These findings describe model behavior on selected static images at a single point in time and do not establish diagnostic accuracy or readiness for clinical use.Larger studies using standardized datasets, repeated model runs, and independent blinded expert evaluation are needed to define the role of vision-enabled LLMs in urologic image interpretation.