22 - Change through the eyes of those who live it: Can Realist Approaches be Participatory and Community-led?

Abstract Information: Realist evaluations enable us to answer critical questions about the programming that we support: what works, how, why, for whom and under what circumstances. By unpacking the stories of community members, local leaders, and program implementers, they can help us understand the responses of these stakeholders and conditions that triggered these responses. Although not commonly sought, this information is critical, particularly for people who design and implement programs. Yet it is crucial to consider which stories are being told, how, by whom, and for what purpose. This roundtable examines whether realist evaluations can be carried out in a manner that not just uses the stories of communities but honors their ownership and voice in these stories. It explores the partnership such a process entails and roles and responsibilities of community members, implementing organizations and evaluators in it. Evaluators and community development practitioners from the Movement for Community-led Development (MCLD), World Vision, The Hunger Project, USAID and the Institute for Development Studies will frame the conversation by sharing their own stories of making realist evaluations more community-led and participatory. Then together with attendees we will delve into the following questions: * What are the methodological strengths and challenges to incorporating community stories in realist and theory-based evaluations? * What would a process that honors the community voices and stories in these types of evaluations look like? How would it impact the process of development of the program theory itself? Who owns the program theory if it is developed through a participatory process? * What burden and legacy do these types of evaluation impose on communities? How can they be reframed to generate learning questions and test program theories with (not from) communities and local stakeholders?· This session is purposefully framed as a roundtable to enable mutual-learning among those that are organizing it and those attending.

Relevance Statement: As the international development and humanitarian sectors recognize the importance of community leadership, the field of evaluation needs to keep pace. Stories are essential for evaluation. Yet it is crucial to consider which stories are being told, how, by whom, and for what purpose. On one hand, this means recognizing new evaluation approaches like Made in Africa evaluations or equity-focused evaluations, many of which are emerging from the Majority World (commonly referred to as Global South). On the other, it necessitates re-examining existing evaluation methodologies and approaches to make them more community-led or at least participatory. This session focuses on the latter. The Realist approach to evaluation enables us to understand how and why certain programs work in specific contexts and for whom. They illuminate the human process of change triggered by program interventions - why do different people respond differently to resources and opportunities provided by programs, and how do their responses bring about outcomes, expected and unexpected? Yet, the methodology attached to this approach is niche and specialized; for example, the distinction between what realists refer to as contexts and mechanisms baffles even those trained in the approach. How, then, can community members participate in a realist-like evaluation process and use this approach to design their own programs better? This session will draw lessons from three different projects seeking to make the realist approach more approachable for locally led programs. Despite the problematic terminology and methodological challenges, realist evaluations have much to offer to the community or locally led interventions. For example, it is equally essential for community members and local actors -as is for funders and implementers to understand that even when programs are designed with or by communities, some of the theories or pathways work as expected, and others don't. Or learn how different groups in their midst may experience and respond to the program differently. A hallmark of realist evaluations is starting with formulating the theory, testing it, and refining it as part of the evaluation process. Thus, providing an opportunity to impart replicable learning at an accelerated pace. Making the realist approach more community-led requires evaluators to simplify the methodology -i.e., to look beyond Context+Mechanism=Outcome (CMO) configurations into making it a way of thinking that enables community members and organizations to explicitly recognize their assumptions and have tools to examine whether or not they worked. This also requires a new type of partnership between communities, implementing organizations, funders, and evaluators. It needs a partnership where communities are not just sources of stories used to evidence causal pathways in a program theory but also decide which stories to explore further and how to absorb and put into practice emerging lessons. Therefore, this session will expand the thinking around realist and theory-based approaches and raise important questions about which stories are told and who decides and tells them. The organizers are experienced evaluators, CLD practitioners and funders who will share how they have adapted such an approach in real-world evaluations and seek input from those participating.