Rater Dependence in Emergency Department Catheter-Appropriateness Audits: A Prospective Three-Rater Study of Item-Level Reliability

Background/Objectives: Catheter-appropriateness audits report a single rate without a measure of its reliability. We measured how far an emergency department (ED) estimate depends on the clinician applying the checklist, and which items carry that dependence.

Methods

We enrolled 117 consecutive adults catheterized in a tertiary ED between 21 January and 21 May 2026. Three emergency physicians independently classified every patient against a ten-item checklist drawn from the 2009 CDC/HICPAC guideline and the Ann Arbor criteria. A designated senior rater served as comparator, a clinical judgment rather than a criterion standard. We computed standalone rates, per-item Fleiss’ κ and Gwet’s AC1, patient-level agreement, and a post hoc counterfactual item-replacement analysis.