Disaster & Emergency Management Evaluation
Gregory Makabila, MBA, Ph.D
Deputy Chief of Party (MEAL) – Resilience Food Security Activity
Catholic Relief Services - Ethiopia, Ethiopia
Maria Alemu, MA Development Studies (she/her/hers)
Collaborating, Learning, and Adapting Advisor -Resilience Food Security Activity
Catholic Relief Services, Ethiopia
Location: Room 208
Abstract Information: Globally, the understanding of poverty has evolved to recognize that it is dynamic rather than static, with high levels of unplanned risks faced by poor households in response to shocks, stresses, and seasonal extremes. In the last decade, there has been a growing emphasis among development partners, governments, and practitioners on resilience, looking at underlying factors that determine the return to equilibrium after a shock by households and communities. Designing and implementing effective resilience programming requires an evidence-based, multidimensional understanding of how communities are affected by and cope with shocks in both the immediate- and long-term, and how this changes over time. Unfortunately, most resilience measurement approaches rely on annual surveys, which fail to capture seasonal shifts or how resilience fluctuates in the period before, during and after a short-term shock. To address this gap, Catholic Relief Services (CRS) began developing the Monthly Interval Resilience Analysis/ formerly known as Measurement of Indicators for Resilience Analysis (MIRA) approach in 2016 in Malawi. MIRA is a multi-faceted methodology that allows stakeholders and communities to better measure and understand resilience, and in turn improve and refine resilience programming. Since its initial development this approach has been scaled to Ethiopia through the Resilience Food Security Activity (October 2021-September 2026) which has leveraged MIRA for context monitoring to provide information of the impact of contextual changes on participants, which can be useful for rapid program adaptations. Modeling results is helping inform development of a contextually sensitive optimal package of interventions. The MIRA protocol consists of seven key elements: 1. Digital Data Collection: MIRA combines a baseline, end line and 15-minute monthly household survey to collect information on context, shocks, food security and coping strategies. Information is collected using smartphones and the mobile application, CommCare. 2. Capacity building & incentives: Enumerators hired from participating communities are trained to collect household information using a digital survey. Enumerators spend approximately 10-15 minutes per interview talking to households about shocks such as drought, flood, diseases, and also those affecting a specific household such as injury or death of a family member. Enumerators are compensated using mobile money. Using community-embedded enumerators helps reduce attrition, ensures there isn't a gap in data collection and facilitates community engagement. 3. Automation: Surveys are uploaded to a cloud server monthly, which allows for remote data collection and monitoring. This feature will be especially useful during the COVID-19 pandemic when travel is limited. 4. Responsiveness: The survey can be adapted to include modules in response to specific shocks and contexts. 5. Quality assurance: Data quality is verified using spot checks and removing outliers. 6. Community engagement: The data is shared with community and district level committees and national partners. 7. Data use: The data is used to make evidence-informed decisions on which interventions to prioritise and how to best spend resources according to the current context. We will demonstrate how this tool is helping optimise project performance by giving high frequency information on shock and their effects on food security status.
Relevance Statement: Having been implemented in multiple countries, MIRA is offering the following value to the field of evaluation: 1. Builds on evidence base for resilience measurement: MIRA has developed into a multi-faceted protocol for measuring and understanding resilience, and for improving and refining resilience programming, due to a few novel features. First, unlike commonly practiced methods of collecting only baseline and end-line surveys, MIRA involves infrequent collection of a broad set of information combined with monthly monitoring of food security and shock experience. It differs from most 'early warning' systems in its panel structure which permits more sophisticated analysis than repeated cross-sectional data. In turn, data are collected using a Computer Assisted Personal Interviewing (CAPI) platform, which allows for near real time updates of the situation on the ground. The resulting data not only facilitate the development of resilience measures and research but have been used by CRS and partners for monitoring, to improve targeting, to adapt programs to suit needs, and, given the design opportunity, for impact evaluation. 2. MIRA reinforces the importance of context monitoring and offers a practical approach to monitor the effect of contextual shocks on livelihoods and resilience of households: MIRA has already proven useful in Malawi in tracking the onset of specific crises, notably flood, drought, and Fall Army worm. Data from the pilot area in Madagascar, in the Grand Sud region (in partnership with the USAID-funded HAVELO project, as well as with independent funds) have likewise been used to inform targeting as well as monitor household wellbeing in the face of numerous shocks, including drought conditions and the COVID-19 measures. In Ethiopia, MIRA feeds into emergency update meetings, a forum through which project staff review context and resilience monitoring data across Ethiopia and make adaptive management decisions. 3. MIRA introduces ways of engaging communities in the practice of evaluation: Through SMS, CRS Ethiopia is sending early warning and guidance on how to respond to shocks to more than 10000 households monthly depending on the priority shocks for the month. Frontline staff have a process of engaging with community early warning committees to determine appropriate actions including changing coping strategies, and advocacy based on MIRA data.