Feminist Issues in Evaluation
Brittany Dernberger, PhD (she/her/hers)
Senior Manager, Systems-Level Impact
CARE, United States
Tamara Jurberg Thomer (she/her/hers)
Project Manager
CARE, New York, United States
Maria Luisa Ramirez, n/a
Manager in International Affairs
GENFAMI, United States
Prabha Paudel, n/a
Project Coordinator
WOREC Nepal, United States
Location: Grand Ballroom 7
Abstract Information: The international development and humanitarian sector increasingly emphasizes decolonization and localization. Yet evaluation practices are often still situated in Global North-centered processes that can be extractive and do little to capacitate local actors. As large international NGOs grapple with how to shift from implementors to conveners, evaluation practices also must shift. Using the example of a USAID-funded initiative to increase the representation of Women-Led Organizations (WLOs) in humanitarian decision-making, we demonstrate a process for decentralizing evaluation and learning activities through a “self-led localization” approach while ensuring globally aligned learning across four countries. Embracing the power of storytelling, mixed methodology, and feminist evaluation practices, we demonstrate how one international NGO – CARE – is working with Women-Led Organizations in Afghanistan, Colombia, Democratic Republic of Congo, and Nepal to engage in participatory action learning and evaluation. The initiative is implemented by a Lead Partner Organization in each country and the evaluation and learning are conducted by a Lead Learning Partner (LLP) in each country. LLPs are established high-capacity Women-Led Organizations and are involved in all aspects of evaluation and learning—from establishing project indicators to data collection, analysis, and dissemination of learning. CARE convenes a cohort with the LLPs to aggregate and synthesize data collection tools and protocols globally, and to facilitate cross-country learning and capacity sharing. Through this distributed, feminist, decentralized approach that embraces qualitative methods and storytelling, we’re seeking to flip typical power dynamics upside down in which a Global North NGO sets the research agenda, designs all the research protocols, and extracts data from partner organizations in the Global South.
Relevance Statement: Our panel engages with storytelling by highlighting evaluation and learning from the Women’s Voice and Leadership in Humanitarian Settings (WHS) initiative. WHS aims to strengthen the voices and leadership of Women Led Organizations’ (WLOs) in humanitarian planning and decision-making through capacity-sharing, learning, and grant-making for WLOs in Afghanistan, Colombia, the Democratic Republic of the Congo (DRC), and Nepal. Equal partnership is a core component of the project design, implementation, and evaluation. The feminist, systems-level evaluation of Women-Led Organizations within the humanitarian structure. This presents several complex theoretical evaluation challenges, which our panel will address: 1. How do you measure systems-level change at a global level through a locally-led approach? What is the role of storytelling in this analysis? 2. How do you embed feminist evaluation approaches within the context of power dynamics between international NGOs and local organizations? 3. How do you employ a feminist, participant-focused evaluation framework in emergency contexts? Storytelling can bridge the local to the global. Storytelling is required to measure systems-level changes and systems-level impact. Some of the most significant systems-level changes are messy and intangible (https://careevaluations.org/evaluation/evaluating-systems-level-change-and-impact-in-cares-programming-in-ecuador-ethiopia-nepal-and-uganda-a-global-report/). As we measure how Women-Led Organizations (WLOs) are engaging with the humanitarian system, an important contribution of our baseline study is that we’re not only comprehensively examining the current organizational capacity of WLOs but also assessing other layers of analysis, such as: • Understanding the nature of humanitarian crises and the diverse needs of women and girls during those crises • How WLOs are currently engaging in humanitarian settings and women’s leadership, capacity and participation to address the crises • Status of financing for response to specific needs of women and girls • Functioning of different policies, mechanisms, committees and networks related to disaster risk reduction and other emergency-related contexts These research questions cannot be answered through quantitative analysis alone. We need qualitative data – storytelling – to assess and measure impact. While the field of evaluation has come a long way in terms of embracing qualitative and developmental evaluations, there is still an underlying narrative that Randomized Control Trials (RCTs) and rigorous quantitative analyses are the gold standard. This year’s theme of storytelling provides a poignant opportunity to question those assumptions, which are often rooted in systems of oppression that prioritize a particular form of data generation. At a global level, it is often funders and evaluators in the Global North who use Global South partners as sites of data generation. Without applying a feminist evaluation lens, and thinking carefully about research agendas and who benefits from the learning, evaluation practices can be quite extractive. Additionally, the way we are approaching this panel presentation itself embraces the theme of feminist collaboration and storytelling. This panel is being submitted by Women-Led Organizations and CARE—an international NGO. While not all four of our country partners will be able to attend AEA in person, we will incorporate storytelling into our panel presentation by including contributions from partners who are not present (i.e., through pre-recorded video messages or other storytelling content).
Presenter: Brittany N. Dernberger, PhD (she/her/hers) – CARE
Presenter: Maria Luisa Ramirez, n/a – GENFAMI
Presenter: Erika Veloza – GENFAMI
Presenter: Cristina Burgos – Surcos de Vida
Presenter: Prabha Paudel, n/a – WOREC Nepal
Presenter: Prasanna Raut – Centre for Women's Awareness and Development (CWAD)