Qualitative Methods
Lauren Serpe
Director, Learning, Evidence and Impact team
Pact, United States
Lauren Serpe
Director, Learning, Evidence and Impact team
Pact, United States
Karen Biesbrouck, n/a
MEAL Team Lead
Oxfam Novib, Netherlands
Kyaw Thura Tun, MSc
Senior M&E Officer - ACHIEVE
Pact
Tripti Pande, MScPH (she/her/hers)
Senior Officer, Learning Evidence and Impact
Pact, United States
Location: White River Ballroom C
Abstract Information: Outcome Harvesting (OH) practitioners are often drawn to the methodology’s flexible nature and its ability to capture powerful stories of change from project stakeholders. It not only captures open-ended reflections from project stakeholders and also provides external validation through knowledgeable, neutral stakeholders. To date, Pact evaluators have implemented ten Outcome Harvesting (OH) or OH -inspired assessments across various projects and countries. However, in adapting the methodology to the needs of projects and the realities of the implementing contexts, we began to wonder, what degree of adaptation is acceptable? To explore this question, Pact undertook a retrospective analysis of its ten OH evaluations to assess what may be considered acceptable adaptations and which strayed too far from the spirit of the OH principles. Each case faced different constraints that influenced how they implemented the six steps of Outcome Harvesting. This panel will share the summary results of this review, followed by detailed presentations of two Pact cases of OH tailoring and one case from Oxfam Novib that will highlight how they tailored OH to address a donor’s concern about bias in the methodology. Each presenter will share the use cases and how they tailored the methodology to programmatic and operational constraints: an HIV prevention and mitigation project in Zambia, an economic and civil society strengthening project in Cambodia, and a policy advocacy program across multiple countries. The audience will gain with an understanding of what may be considered acceptable tailoring of the OH methodology and which adaptations of the methodology may render the exercise less rigorous. We will also foster a vibrant discussion portion of the session to hear other practitioners’ experiences in tailoring the method and obtain their input on our initial conclusions.
Relevance Statement: Outcome harvesting (OH) is a participatory evaluation approach that offers excellent flexibility to measure complex program outcomes. It is an open-ended, reflective method that asks project beneficiaries to tell their story of change and how participation in a project has contributed to these results. The method promotes probing for unintended and potentially even negative results they have experienced as a result of participation in the project. While the method is intentionally flexible, it still requires fidelity to nine principles of engagement and the specified six steps. International development programs often operate in complex operating environments and pursue outcomes that predetermined metrics are unable to predict. In addition, these programs often face funding and timing challenges that require tailoring of assessment methods. In most of Pact’s eight use cases of Outcome Harvesting across seven countries, evaluators tailored the method in various ways—meaning their implementation of the six steps and nine principles varied from the official Outcome Harvesting guidance. Upon reflecting on five years and eight use cases of using Outcome Harvesting at Pact, we decided it was important to consider how to effectively tailor the method to make it useful for projects without losing methodological rigor. This panel will share findings from a study of tailoring OH use cases and propose criteria for what can be considered acceptable adaptations and what cannot. The first panelist will summarize the Pact study findings, the second and third panelists from Pact will share details of their specific cases, and the final panelist from Oxfam Novib will present their case of tailoring OH to reduce the perception of bias in outcome statements. These cases were included in this panel because they represent three different types of international development programs (health, civil society strengthening, and policy advocacy) and they differ in the ways they modified the OH method. This panel will be most useful for practitioners with some working knowledge of the OH method. Attendees will leave the session with an understanding of how to acceptably modify the OH method and which pitfalls to avoid. The panel is submitted to the Qualitative Methods TIG as it aligns directly with the TIG’s mission and values of sharing qualitative methods practice.
Presenter: Chhork Boeurng, MDM – Pact
Presenter: Thomas Mugala, n/a – Pact, ACHIEVE
Presenter: Lauren Serpe – Pact
Presenter: Karen Biesbrouck, n/a – Oxfam Novib