How the Making it Work Team adapted its program theory for its Realist Evaluation survey

 

The Making it Work Study Team has developed a Working Program Theory that has adapted the Realist Evaluation approach to include Indigenous Ways of Knowing and Doing. This including using the medicine wheel and thinking about our context (C), mechanism (M), and outcome (O) statements as spirals, not linear as usually happens with Realist Evaluation:

You can read more about the process of building the Making it Work program theory in our 2020 CAHR poster: Using a Two-Eyed Seeing Approach to Realist Evaluation in a Community-Based Research Project and how we used it for our qualitative data collection work in our 2022 CAHR poster – Learning Together: Analysis Through Differing Perspective in the Making it Work Study.

As an introduction to the survey we recorded this narrated presentation from Sherri Pooyak, one of the Principal Investigators on the study:

We had five areas of inquiry for our survey. These emerged as areas with less clarity or discordance after doing initial analysis from our qualitative data collection. They were harm reduction; harm reduction and culture; cultural safety, peer-led service and peer workers; and linking case management and community development services.

As mentioned, one of the areas we wanted to explore in our survey was harm reduction and culture. Our original spiral looked like this:

After working with our Study Team we decided to simplifying it and make it more accessible for the survey:

We then worked with the Study Team and Elder, we built off the experiences of our case study site to develop a story that would help to illustrate this spiral. You can listen to Martha’s story about harm reduction and culture here:

After either listening to the story or looking at the picture (many people felt they needed to do both), the survey participants were asked the following questions. With many people either taking the time themselves or working with a study team member to add in thoughtful answers in the open-ended sections.

Please rate your agreement on: (1-5 Likert scale) Strongly disagree/disagree/neutral/agree/strongly agree/don’t know/prefer not to answer

This story/picture represents….

  Strongly Disagree Disagree Neutral (neither disagree nor agree) Agree Strongly Agree Don’t Know Prefer Not to Answer
A good way harm reduction services can work at organizations              
How services work at this organization              
How services work for Indigenous people              
How services work for non-Indigenous people              
 The ways harm reduction services are set up or run (the “if…” statements)              
 The changes harm reduction services make for people (the “then…” statements)              
 The things an organization does to make harm reduction services work (the “because…” statements)              

What are your reflections on the different steps in the story and the picture above? Do they represent how harm reduction services work at organizations? What is missing? What is different from how you see harm reduction services working in real life? (open-ended)

How do you think the COVID-19 pandemic has impacted harm reduction services?

 

Questions? Feedback? Get in touch!
Joanna Mendell, Director of Research and Evaluation, [email protected]