Environmental Program Evaluation
Christina Seybolt, MUP (she/her/hers)
Senior Technical Specialist
Social Impact, Inc.
Washington, District of Columbia, United States
Mike Duthie, MSc
Senior Technical Advisor
Social Impact, Inc., United States
Kimberlee Chang, PhD
Research Manager
Social Impact, Inc., United States
Heather Huntington, PhD
Professor of Practice, Associate Director, DevLab@Penn
University of Pennsylvania, United States
Location: Room 205
Abstract Information: The United States Agency for International Development (USAID) has long recognized the importance of cross-sectoral, integrated programming to advance sustainable development. USAID’s Health, Ecosystems, and Agriculture for Resilient Thriving Societies (HEARTH) program co-creates public-private partnerships to advance integrated investments in conservation, agriculture, health, governance, and other sectors. Despite a growing body of literature, the evidence base for understanding whether integrated programs are working, why, and for whom, is lacking. Generating rigorous evidence about how human well-being and conservation/biophysical outcomes impact each other—and what conditions might contribute to the realization of positive outcomes—is of primary interest not just for USAID, but the broader development sector. To fill this gap and improve integrated program and policy effectiveness, evaluations are in progress for HEARTH activities in Zambia and West Africa. This session will discuss important lessons learned for evaluators to consider when designing and conducting evaluations for cross-sectoral programs, including the potential benefits for donors and implementing partners, as well as ways to overcome common challenges.
Relevance Statement: Relative to other development sectors, there is a dearth of counterfactual-based evaluations of conservation programs (Ferraro 2009). However, frequently cited challenges such as large variability in outcomes, long time lags between intervention and expected results, programs with multiple interventions, complex spillover effects, and large spatial scales are not unique to conservation programs. The same can be said for cross-sectoral programs, which face additional challenges given the expanded set of human well-being and conservation/biophysical outcomes of interest. In this case, the stories we tell ourselves as evaluators and practitioners about these challenges create a false narrative that limits learning about how to increase program effectiveness, ultimately stifling innovation. At their core, cross-sectoral programs are based on the assumption that human well-being and conservation/biophysical outcomes are linked and have the potential to impact each other positively. For example, as human well-being improves, there might be a reduced need for extractive/unsustainable behavior, such as lower demand for forest resources during times of stress or shocks. Alternatively, as biophysical outcomes improve, this might improve agricultural productivity or human health. However, in many instances, the opposite may be just as likely to occur. For example, as human well-being improves, there might be greater demand on natural resources for things like energy, building materials, etc. Alternatively, as biophysical outcomes improve, this might occur at the expense of human well-being, as communities are barred from accessing resources which they previously relied on for their livelihoods with few available alternatives. Generating rigorous evidence about these relationships and how human well-being and conservation/biophysical outcomes impact each other—and what conditions might contribute to the realization of positive outcomes—is of primary interest not just for USAID, but the broader development sector. We draw from experience designing quasi-experimental evaluations for two cross-sectoral programs, one in Zambia and one in West Africa, to provide real-world examples of how these challenges can be overcome. We also provide examples of benefits resulting from the evaluation design process itself, such as promotion of adaptive management which led to greater insights into specifics of implementation plans and the extent to which the planned interventions were in line with the activities’ theory of change and desired impacts.
Presenter: Kimberlee Chang, PhD – Social Impact, Inc.
Presenter: Christina Seybolt, MUP (she/her/hers) – Social Impact, Inc.