Collaborative, Participatory & Empowerment Evaluation
Abdoul Karim Coulibaly, n/a
Monitoring & Evaluation Consultant
Freelance, Canada
Yulianto Dewata, Director
Monitoring & Evaluation Practitioner
PT. Reforma Visi Utama, Indonesia
Steve Powell
Director
Causal Map Ltd
Clevedon, England, United Kingdom
Marina Apgar (she/her/hers)
Research Fellow
Institute of Development Studies - Centre for Development Impact, England, United Kingdom
Location: White River Ballroom A
Abstract Information: Outcome Harvesting is a method for telling evaluative stories about what has emerged from an intervention in a complex context. It is powerful in its ability to discover not only the predicted outcomes, but also what else is emerging and why. It is a storytelling tool that is often deployed in deeply participatory ways and can make visible many different voices and ways of understanding what is changing. At its best, it is not only about the outcomes: it is also about deep exploration of how and why those outcomes emerged - of the causal pathways leading up to them. This session will share three stories from very different types of initiatives in different contexts and geographies, each of which was implemented with careful attention to understanding the causal pathways of change. Join us on a journey through Nigeria, Liberia, and Indonesia, as we explore the method and how we approached making causality visible. Learn about tools that can help in the analysis and visualization of causality as part of an Outcome Harvesting study. This session is part of the Causal Pathways Initiative’s ongoing work to build understanding, will, and skills to more fully discover causality amid complexity.
Relevance Statement: One of the two most often downloaded articles from Foundation Review in 2022 was “Lost Causal,” an article that challenged philanthropy and evaluators to reconnect to the importance of understanding how change is really happening, not just predicting what might happen and looking for evidence of it. A year later, over 3,000 people signed up for an SSIR webinar on the same topic - how can we understand how change really happens in complex, dynamic systems (without resorting to experimental and quasi-experimental designs that are often not appropriate). The Causal Pathways Initiative is a new effort that is seeking to respond to this outpouring of interest - the need expressed by commissioners and evaluators alike. The network of methodologists, evaluators, and funders who are part of the initiative identified a need to build awareness, understanding and will to use causal methods, alongside a significant need to address skill gaps in the field. The network members have concluded that a fundamental problem is the frequency with which these methods are implemented without rigor and deep participation of marginalized voices. In response, one of the actions the network is taking is to speak in front of commissioners and evaluators in many different venues, beginning with the work they are already engaged in and helping make visible how causal understanding can be strengthened in that work. To this end, this session is focused on a method many are using in complex, dynamic strategies and a method many others have been asking about in the context of the casual pathways initiative: Outcome Harvesting. It is a method that can help understand causality when it is implemented with deep attention not just to the outcomes that emerged, but also how, why, and under what conditions they came about. This causal understanding is strengthened by participatory involvement in discovering the pathways to change, interpreting from multiple perspectives in the system. Too often, however, we see evaluations using outcome harvesting that describe and validate the presence of observed changes without investigating how or why they occurred, including the relationships between what was funded, what was implemented, and what resulted. Even more problematic, we see evaluations that simply assume, without investigation, that relationships exist between implemented strategies and observed outcomes. We reached out to leaders in the outcome harvesting space and confirmed that the other sessions members of that network are submitting will not be prioritizing this aspect of outcome harvesting and they see value in this content being brought forward at AEA. Examining causal relationships is not just a necessary tool for understanding how change is happening – it is critical for testing our assumptions, challenging philanthropic theories of change, engaging expanded voices in understanding where, how, and why change is happening, and ultimately increasing our impact on complex, systemic issues. This issue is timely – both because of the increased attention over the last year as well as its close alignment with a field shift toward greater focus on equity, participation, and power shifting.
Presenter: Abdoul Karim Coulibaly, n/a – Freelance
Poster Presenter: Marina Apgar (she/her/hers) – Institute of Development Studies - Centre for Development Impact
Presenter: Yulianto Dewata, Director – PT. Reforma Visi Utama