The Making it Work project is an Indigenous-focused, community-based research project that uses an Indigenized ‘realist evaluation’ approach. The study aimed to understand what works best for people living with HIV, hepatitis C, and/or challenges with mental health or substance use, who are accessing community-based services.
Realist Evaluation is designed to go beyond asking ‘if’ a program works, instead asking how, why, when, and for whom services work well. The project team developed a program theory and adapted the realist evaluation processes, which highlighted the ongoing relational and evolving nature of these services. This was done by using the Medicine Wheel as a visual representation of the important aspects of their programming.
To confirm the program theory, the team used a variety of data collection approaches. This resource focuses on the survey process. Realist Evaluation surveys are complex and are not easy to develop, they are meant to ask about very complex ideas that are hard to translate into simple survey questions. Using a two-eyed seeing approach also meant exploring new ways of developing a survey. It is important to share learnings and further work should be done with this form of survey development.
In this resource, we highlight key approaches to developing and conducting a survey using an Indigenized approach that came from reflections of the survey development and implementation:
- Storytelling
- Relationships & reciprocity
- In-person data collection
- Supports for surveyors and participants
- Flexibility and adaptability
In some ways this approach to survey development was successful and some participants spoke about how they liked having a variety of tools available for them to engage with the ideas. However, the study team felt that the goal of asking realist evaluation questions remained challenging and imperfect in a survey format and would require further adaption because of the complex and nuanced nature of the questions that need to be asked.
Questions? Feedback? Get in touch!
This post was prepared for PAN’s Research and Evaluation Treehouse by:
Leanne Zubowski, Research and Evaluation Specialist, [email protected]