Advocacy evaluation is a growing field of evaluation that seeks to capture a lot of the work that often remains unaccounted for, despite its importance for promoting equity and social justice. The theme of this year’s Canadian Evaluation Society Conference focused on evaluation utilization. How do we make sure that we not only use evaluation to inform programs and services but that we also use it to promote social change? This question was at the forefront on my presentation Measuring impact of advocacy – from theory to practice of change.
Most nonprofit organizations are involved in advocacy in addition to providing their services and programs. The deep knowledge of the issues and the recognition of systemic barriers are some of the reasons organizations engage in advocacy. Advocacy can be defined as both a goal, i.e., supporting or promoting a certain cause, and a set of strategies that can include campaigning, awareness raising, policy and coalition work. The importance of advocacy efforts can hardly be overestimated, and evaluation is one of the ways to demonstrate its value.
Advocacy evaluation is different from program evaluation in several key ways:
1. Advocacy evaluation is messy. Its messiness stems from the unpredictable nature of change that is often influenced my multiple players and circumstances.
2. Advocacy is not a linear process, and we need to shift from cause-effect style thinking that is often used in theories of change and logic models, to determining the extent to which certain factors caused a change (attribution analysis) and cumulative evidence. A list of factors that need to be considered in advocacy evaluation can be visually represented as in this figure below. Different advocacy related activities (e.g., public education, media advocacy, communications, policy analysis) are shown in the area where the x-axis represents Audiences (Public, Influencers, Decision Makers) and y-axis – Levels of engagement (Awareness, Will, Action).
Source: Gienapp, A., Lynn, J., Ochtera, R., Raynor, J. Unique Methods in Multi-Stakeholder Advocacy Evaluation. TCC Group. https://www.tccgrp.com/resource/unique-methods-in-multi-stakeholder-evaluation/
3. Most advocacy efforts are time consuming and long-term. Advocacy evaluation needs to account for that.
4. Advocacy requires collective efforts. It usually takes coalitions to work on a specific advocacy issue.
5. Advocacy is multi-layered, and we need to be clear about what it is we want to evaluate. We might want to evaluate advocacy capacity that includes indicators like the level of organizational readiness, institutionalized support for advocacy, organizational commitment to and resources for advocacy. We might also want to consider evaluating advocacy network with a focus on coalition strength/cohesiveness and/or relationships with decision makers.
All of these unique characteristics of advocacy evaluation require different approaches to data collection. Learn more about different methods of advocacy evaluation in Part 2 of this blog series coming soon!
Learn more about PAN’s advocacy work in the Policy change and collective action section.
Questions? Feedback? Get in touch! Alfiya Battalova, Evaluation Manager, [email protected]