Recommendations for addressing methodological challenges in evaluating a contact tracing programme during the COVID-19 pandemic: A Commentary

Abstract Rigorous and comprehensive evaluation of public health interventions can identify adjustments and whole-scale changes to improve programme effectiveness in real time and/or for the future. However, during public health emergencies, designing and implementing quality programme evaluation is often deprioritised or overlooked altogether, leading to missed opportunities to set up the data collection, data sharing and record retention systems that facilitate high-quality, efficient and effective evaluation. We conducted a comprehensive mixed-methods evaluation of a COVID-19 contact tracing programme during an acute phase of the COVID-19 pandemic on a programme with multiple stakeholders, objectives and data systems that were mobilised during a chaotic time. To inform future programme evaluations conducted about and during emergency responses, we conducted a retrospective assessment of our evaluation process by reviewing documentation of our methods and the information the programme provided, conducting retrospective team debriefings and reviewing our responses to evaluation challenges against relevant methodological literature. The assessment identified key methodological challenges encountered during the evaluation, the strategies we used to address them, and practical recommendations for future evaluators. Rather than evaluating programme outcomes, this paper reflects on the conduct of the evaluation itself and identifies practical lessons for future emergency programme evaluations.