Mixed Methods Process Evaluation of a Sanitation Behavior Change Intervention in Rural Odisha, India

Abstract

Background

Process evaluations of public health programs are critical to understand if programs were delivered as intended and to identify improvements for future implementations. Here we present a mixed methods process evaluation of the Sundara Grama intervention , which sought to improve latrine use and safe child feces disposal among latrine-owning households in rural Odisha, India.

Methods

The Sundara Grama intervention was delivered to 36 villages in Puri district by a grassroots non-governmental organization (NGO) and included eight activities: palla performance, transect walk, community meeting, community wall painting, mother’s meeting, positive deviant household recognition, household visit, and latrine repairs. The process evaluation quantitatively assessed fidelity, dose delivered, and reach, and qualitatively examined recruitment, context, and satisfaction. Quantitative data collection included an activity observation survey, activity record, and endline trial survey. Qualitative data collection included an activity observation debrief and in-depth interviews with NGO mobilizers. For the quantitative data, a ‘delivery score’ was calculated for each activity, as well as the proportion of target participants in attendance. Qualitative data were analyzed using thematic analysis.

Results

Mean delivery scores, reported as a percentage, were moderate to high. Household visit activities (97% general visit, 96% positive deviant visit) and the mother’s meeting (81%) had the highest delivery scores, followed by the palla (77%), transect walk (77%), and community meeting (60%). Activities were attended, on average, by 30% to 73% of latrine-owning households. Several factors aided delivery, including pre-intervention rapport building visits and village stakeholder support. Factors that hindered delivery included inclement weather, certain recruitment strategies, and village social dynamics.

Conclusions

Overall, the Sundara Grama intervention was implemented as intended and achieved good reach. The findings suggest education-entertainment strategies, like the palla, and multi-level communication approaches are particularly beneficial. The results also showcase the importance of examining the implementer experience and broader context.