Guideline Concordance of Antibiotic Recommendations From Large Language Models in Dentistry: A Vignette-Based Evaluation in Georgian and English

Introduction: Inappropriate antibiotic prescribing in dentistry is a recognised stewardship concern.Large language models (LLMs) are readily accessible sources of prescribing information, but dental evaluations have been conducted almost entirely in English.We compared the guideline concordance of matched Georgian and English LLM recommendations for common dental presentations in a setting in which we could identify no national dental antibiotic prescribing guideline.Materials and methods: In this preregistered vignette study, 20 standardised scenarios (10 antibioticindicated, 10 not) were submitted in Georgian and English to the free-tier interfaces of Claude (Anthropic, San Francisco, USA), ChatGPT (OpenAI, San Francisco, USA) and Gemini (Google LLC, Mountain View, USA) on three occasions (360 responses).Responses were normalised, blinded to system and run, and coded by two dentists against a key based on Scottish Dental Clinical Effectiveness Programme (SDCEP), American Dental Association (ADA) 2019 and American Heart Association (AHA) 2021 guidance on five domains: decision to prescribe, agent, dose and frequency, duration, and referral advice.Disagreements were adjudicated and the regimen domains were re-scored under clarified rules.The primary outcome was full concordance on all applicable domains in the first run, compared across 60 matched scenario-system pairs with the exact McNemar test.Results: Full concordance was 57/60 (95.0%, 95% CI 86.3 to 98.3) for English and 46/60 (76.7%, 64.6 to 85.6) for Georgian prompts, a paired difference of 18.3 percentage points (95% CI 7.5 to 29.2; p = 0.003); 12 pairs were concordant in English only and one in Georgian only.The decision to prescribe was concordant in all 360 responses and referral advice in 59/60 per language.In antibiotic-indicated scenarios, an accepted firstchoice agent was given in 30/30 English and 21/30 Georgian responses (p = 0.004); dose and duration concordance was also lower in Georgian (16/27 and 15/27 versus 26/27 and 25/27) but was scored discordant whenever the agent was discordant.All 30 no-antibiotic pairs were concordant in both languages.The between-system comparison was not significant (Cochran's Q p = 0.14).The agent code was reproducible across three runs in 29/30 English and 26/30 Georgian antibiotic-indicated cells (p = 0.25).No first-run response met a prespecified safety-critical error criterion (0/60 per language, 95% CI 0.0 to 6.0).Conclusions: Decision concordance was preserved across languages, but full concordance with a UK stewardship standard was lower for Georgian prompts, chiefly because the first-choice agent was broader than the reference accepts.LLM-generated regimen details should be checked against an authoritative source.The findings underscore the value of accessible, authoritative Georgian-language dental prescribing guidance.