The Devil’s in the Detail: Implementation Fidelity in Evaluating a School-Based Prevention Programme for Children Under 12

Abstract

Purpose
This paper examines implementation fidelity (IF) and the underpinning conceptual framework drawing on the evaluation of a UK-wide, manualized child abuse and neglect prevention program for elementary schools. We describe and assess our approach to assessing IF and consider how IF can inform program development.

Method
We drew on the literature on program fidelity and critical components of the program evaluated to identify three dimensions of IF: Coverage, Quality and Context. Data was collected through external observations using systemized observation schedules which were extracted to be scored using scoring protocols for each intervention type. Scores were calculated by two researchers with a random sample cross-checked by a third member of the research team.

Results
Observation analysis demonstrated consistency in the coverage of content when delivering assemblies for both younger and older children with at least 76% coverage of content across the assemblies. However, observation analysis revealed greater levels of variability in the delivery of workshops. Material on sexual abuse was less fully covered and children reported that some facilitators lacked confidence and clarity in delivering this material (Stanley et al., 2023).

Conclusion
Our results indicate the usefulness of systemized observations in capturing coverage of content, these findings underscore the importance of developing scoring protocols and training observers prior to evaluating program delivery. We highlight the significance of integrating implementation fidelity training for program facilitators and implementers to both assist with monitoring and to maintain quality, despite variations in the actual delivery and setting of the program.