Health Evaluation
Jessica Fehringer, PhD (she/her/hers)
Director
Data for Impact, United States
Jessica Fehringer, PhD (she/her/hers)
Director
Data for Impact, United States
Tory Taylor, MPH (she/her/hers)
Technical Director
Data for Impact
New Orleans, Louisiana, United States
Justus Uzim, n/a
Project Management Specialist (M&E)
USAID/Nigeria, Nigeria
Location: White River Ballroom G
Abstract Information: The USAID-funded Data for Impact (D4I) project is weaving a complex story about activity sustainability under its comprehensive, prospective mixed-methods portfolio evaluation of four USAID/Nigeria Health, Population, and Nutrition (HPN) activities in three case study states. D4I uses data triangulation with diverse methods. These methods include sustainability benchmark development and tracking; most significant change (MSC) method workshops followed by story verification using District Health Information System 2 (DHIS2) data and process and program monitoring data; organizational network analysis looking at local organizations’ connectedness, relationship quality, and other relationship characteristics; key informant interviews with health facility staff; and focus group discussions with community members and local health governance bodies. These data converge to tell the story of the four activities’ relationships and sustainability elements among multilevel government partners and local and international implementing partners to understand, strengthen, and leverage key organizational networks and institutional processes to sustain health programming and improved outcomes. This session will describe the programmatic and outcome sustainability framework and assessment approaches as well as the summary sustainability results to date. It will also share the challenges and successes of crafting and sharing a data-rich, intricate, and real-time story with stakeholders as it evolves.
Relevance Statement: Global health funding initiatives and programs increasingly strive to drive social justice through the promotion of local ownership and sustainability where sustainability extends beyond financing to include capable local institutions with technical expertise and multilevel network access. Evaluation practitioners need to have tools to measure progress toward local ownership and sustainability. However, there is no agreed upon sustainability measurement framework, terminology on the topic varies, and there are methodological challenges to the measurement of sustainability. This panel will showcase an exemplary portfolio-level evaluation design of real-world public health activities that have proven useful to program managers and donors. Presenters will describe how organizational network analysis, sustainability benchmarking, most significant change (MSC) method with story verification, and key informant interviews and focus group discussions are being used in a longitudinal mixed-methods evaluation of three distinct health program models operating at the state level in Nigeria. They will also share analysis results and discuss the evolution of the researchers’ thinking about network measures as potential proxies for program sustainability, setting sustainability benchmarks, and adjusting MSC methodology for the purpose of understanding sustainability. Participants will leave the session with ideas and tools for incorporating network analysis, sustainability benchmarking, and MSC method into their own stories about the interplay between community-based and other health program actors and countries’ progress toward institutionalized solutions to health and development challenges. Evaluators can also apply lessons learned from this panel to advocate with donors and program managers on design decisions related to sustainability. Lastly, panel presentations can raise awareness among program managers and donors on the design challenges that evaluators face in measuring sustainability and in telling stories involving data triangulation with large amounts of ongoing data collection. Such awareness can increase understanding of why certain design and dissemination decisions are made, easing communication between evaluators, program managers, and donors.
Presenter: Siân Curtis, PhD – Data for Impact
Presenter: Huyen Vu, MPH – Data for Impact
Presenter: Justus Uzim, n/a – USAID/Nigeria
Presenter: Tory M. Taylor, MPH (she/her/hers) – Data for Impact
Presenter: Jessica Fehringer, PhD (she/her/hers) – Data for Impact
Presenter: Tory M. Taylor, MPH (she/her/hers) – Data for Impact
Presenter: Siân Curtis, PhD – Data for Impact
Author: Devin J. Cornell, PhD