Systems in Evaluation
Carolina De La Rosa Mateo, MPH (she/her/hers)
Project Manager
Causal Pathways, United States
Marina Apgar (she/her/hers)
Research Fellow
Institute of Development Studies - Centre for Development Impact, England, United Kingdom
Jewlya Lynn, PhD (she/her/hers)
Founder
PolicySolve
Denver, Colorado, United States
Location: White River Ballroom B
Abstract Information: This session is designed to bring to life the practice of participatory causal analysis, recognizing that making visible how and why systems are causing harm, and telling the stories of how and why change is happening when people take action, is more than just evaluation. It is a way to transform the system by combining the storytelling tools of evaluation, the power and insight of those most affected by the system, and the willingness of many different actors to seek to change the system. The presenters will bring to life the CLARISSA project, combining their own storytelling about the project and evaluation elements with videos of children and their advocates talking about how participatory action research is helping them tackle the worst forms of child labor. The chair and discussant will help unpack how the CLARISSA story can help us combat some of the myths we hold in the evaluation field about when and where we can investigate causality deeply, including myths about the appropriateness of causal analysis when doing participatory, equity, and systems-focused work. This session is part of the Causal Pathways Initiative’s ongoing work to build understanding, will, and skills to more fully discover causality amid complexity.
Relevance Statement: One of the two most often downloaded articles from Foundation Review in 2022 was “Lost Causal,” an article that challenged philanthropy and evaluators to reconnect to the importance of understanding how change is really happening, not just predicting what might happen and looking for evidence of predicted outcomes. A year later, over 3,000 people signed up for an SSIR webinar on the same topic - how can we understand how change really happens in complex, dynamic systems (without resorting to experimental and quasi-experimental designs that are often not appropriate). The Causal Pathways Initiative is a new effort that is seeking to respond to this outpouring of interest - the need expressed by commissioners and evaluators alike. The network of methodologists, evaluators, and funders who are part of the initiative identified a need to build awareness, understanding and will to use causal methods, alongside a significant need to address skill gaps in the field. The network members have concluded that a fundamental problem is the frequency with which these methods are implemented without rigor and deep participation of marginalized voices, of those in the system whose experiences are often hidden. Some of the myths explored in both the article and SSIR webinar focus on misperceptions many commissioners and evaluators hold about the incompatibility of rigorously exploring causality in participatory, equity-focused, and systems-change focused strategies. This session is designed to directly combat that set of myths by telling a powerful story, bringing in many different voices by leveraging the presenters and videos prepared by participants in the work. It is a session that can inspire evaluators to see the value of deep investigations of causality - how, why, and under what conditions change is really happening. When we support people to understand causality, we are part of the change story, not just telling it. This issue is timely – both because of the increased attention over the last year as well as its close alignment with a field shift toward greater focus on equity, participation, and power shifting. This issue is also very closely tied to the conference theme. The CLARISSA project is using methods that create narratives about people’s lives and how systems influence them. These narratives are more than stories - they are tools for action. The evaluation helps discover stories, and the stories help shape the evaluation and the strategies to change the child labor systems. Stories fuel change.
Presenter: Jewlya Lynn, PhD (she/her/hers) – PolicySolve
Presenter: Marina Apgar (she/her/hers) – Institute of Development Studies - Centre for Development Impact