Guideline-Concordant Answer Quality of Five Large Language Models on Indication Criteria for Total Hip Arthroplasty in Hip Osteoarthritis: A Single-Run Benchmark Against the 2021 German S3 Guideline (EKIT-Hip)

Background/Objectives: Hip osteoarthritis (OA) is a highly prevalent musculoskeletal disorder associated with substantial functional impairment and healthcare burden. Although evidence-based clinical practice guidelines exist, including the 2021 guideline of the German Society for Orthopaedics and Trauma Surgery (DGOU), their implementation in routine practice remains incomplete. Large language models (LLMs) are increasingly consulted for guideline-related information, but the concordance of their answers with guideline recommendations is uncertain. This benchmark study evaluated the guideline concordance, comprehensiveness and length of answers generated by five widely used LLMs to questions derived from the 2021 German S3 guideline on evidence- and consensus-based indication criteria for total hip arthroplasty in hip OA (EKIT-Hip; AWMF register no. 187-001; lead society DGOU). It did not evaluate the effectiveness or safety of clinical decision support.