Evaluation is a foundation of our work at PAN, embedded in our strategic directions, values and approaches. We are responsible to members and allies to work on the issues and directions that concern our communities. We evaluate informally through conversations and meetings, and formally through surveys, interviews, needs assessments, and other tools. Our Organizational Stigma Assessment Cycle project (OSAC) is no exception to this practice.
OSAC grew out of informal evaluation in discussions at PAN events about stigma advocacy. In exploring different kinds of stigma, it was suggested that a tool be developed to learn about where stigma might be experienced at community organizations. Spurred on by this idea, we looked into organizational audits and accreditation, which led to follow-up conversations on community response, support and readiness. We got excited about shaping this work as a learning cycle, seeing OSAC as a test pilot to learn where and how organizations may unintentionally be contributing to experiences of stigma and where there is need for change.
Because OSAC is a pilot limited to a few test sites, we want to optimize our learning in preparation for a wider launch to follow. We have decided to use a developmental evaluation approach, which is a responsive way of using evaluation to adapt project progression to meet its goals. Read on for an introductory overview of developmental evaluation.
Basics of Developmental Evaluation
Michael Quinn Patton first suggested developmental evaluation as an approach for new work that’s not finalized and/or is taking place in complex environments that change quickly. Developmental evaluation is useful when trying something new as it can maximize the potential and growth of a project. It is responsive in real time, rather than evaluating at the completion of the project. This flexibility can make changes to meet the needs of a developing project or program. As Patton explains,
[Traditional] “Formative evaluation has come to designate any evaluative efforts to improve a program. Improvement means making it better. In contrast, developmental evaluation focuses on adaptive development, which means making the program different.” (State of the Art and Practice of Developmental Evaluation, p. 4, retrieved July 2023)
Traditional evaluation has evaluators maintaining a distance from a project, so they don’t influence or bias the outcomes. In developmental evaluations, evaluators are partners in project progress; gathering and analyzing data to share with partners throughout. Using what’s learned in evaluation, changes and adaptations can be made to improve likelihood of successfully reaching project outcomes.
In supporting systems change, developmental evaluation approaches acknowledge the relationships that exist between many elements in a system. OSAC is designed to support organizations to look at intersectional stigma in their policies, programs, and resources; as well as how they connect with other organizations. The relationship between these building blocks must be examined relationally to understand where they may contribute to stigma.
With OSAC being a new project, we want to know the details of where it does and doesn’t work. With developmental evaluation, we can dig into details along the way, and work to improve our process and resources. Our evaluation team will include PAN team members and pilot organization partners, so we have multiple points of view and participation in how to use evaluation information to strengthen the learning cycle.
We have finished the drafts of the complete learning cycle guides, which are now out for review with our Advisory Committee. We look forward to next steps and will keep you up to date as we move along. If you’d like to know more in the meantime, visit the project page, or contact one of the project coordinators: Janet Madsen or Leanne Zubowski.
Learn more
Brief Introduction to Developmental Evaluation (3-minute watch)
State of the Art and Practice of Developmental Evaluation
Developmental Evaluation: A Short Introduction
Thank you to the Vancouver Foundation for funding this project to address systems change.
Logo designed by Nahanee Creative