13 - Making causality visible amid complexity: Approaches and tools for causal understanding that can be applied in complex, dynamic contexts

Abstract Information: A growing suite of methodological approaches make causal analysis possible in complex, dynamic contexts where the more widely known causal approaches (experimental and quasi-experimental designs) are not appropriate. Causal methods can challenge or strengthen the often uncontested beliefs that underlie interventions while offering evidence about enabling contexts and system drivers. Strong causal analysis considers not only the funder’s model and assumptions, but also the beliefs others hold about how and why change occurs, opening the door to more equitable and less biased ways of understanding change. This full day training session will prepare participants to better understand when and how to integrate a deep causal understanding into an evaluation design. It will clarify how thinking about causality predictively (e.g. in upfront theories of change) is different from analyzing and understanding how, where, and why change is happening. During the first half of the day, participants will learn from stories and cases about the core concepts, including understanding how this work can be done in deeply participatory and equity-focused ways. In the second half of the day, participants will be invited to “choose their own adventure” and more deeply examine a variety of specific methods and approaches, with experts from the field helping them to think through the application of outcome harvesting, QUIP, most significant change, contribution analysis, process tracing, and participatory approaches when doing causal work. Finally, the session will wrap up with guidance on how to implement causal approaches with deep attention to the quality of the research, including understanding what rigor looks like when examining causal pathways. This session is part of the Causal Pathways Initiative’s ongoing work to build understanding, will, and skills to more fully discover causality amid complexity.

Relevance Statement: One of the two most often downloaded articles from Foundation Review in 2022 was “Lost Causal,” an article that challenged philanthropy and evaluators to reconnect to the importance of understanding how change is really happening, not just predicting what might happen and looking for evidence of predicted outcomes. A year later, over 3,000 people signed up for an SSIR webinar on the same topic - how can we understand how change really happens in complex, dynamic systems (without resorting to experimental and quasi-experimental designs that are often not appropriate). The Causal Pathways Initiative is a new effort that is seeking to respond to this outpouring of interest - the need expressed by commissioners and evaluators alike. The network of methodologists, evaluators, and funders who are part of the initiative identified a need to build awareness, understanding and will to use causal methods, alongside a significant need to address skill gaps in the field. The network members have concluded that a fundamental problem is the frequency with which these methods are implemented without rigor and deep participation of marginalized voices, of those in the system whose experiences are often hidden. Many of our other proposed sessions at AEA are focused on building will, but the participants in the SSIR article and evaluators we met with at AEA last year have continued to emphasize the importance of having opportunities to build understanding and skills in designing and implementing evaluations that can tell rigorous causal stories. This pre-session training is designed to address exactly that need. The first half of the day is focused on an overarching understanding of the approach, including when it is needed and how to design evaluations to answer causal questions. The second half of the day is a “choose your own adventure” style exploration of multiple methods and approaches that evaluators might apply within a larger causal pathways approach. This directly responds to the requests we've been hearing from evaluators to learn about specific methods and how to engage in participatory ways of exploring causal pathways. Our hope is that evaluators will be more confident in their ability to begin using this approach, along with having a sense of peers and familiarity with the resources available through the Causal Pathways Initiative.