Outcome harvesting (OH) allows the measurement of program outcomes in complex programs such as a global policy advocacy program that aims to change social actors´ behavior in the public and private sector. Similar to the experiences at Pact, the use of OH was also adapted by a Dutch NGO alliance to address a bias concern from its funder.
To increasingly contribute to systemic change, the Dutch Ministry of Foreign Affairs’ portfolio has increasingly included programs focused on policy advocacy. However, the realities of evaluating such programs has challenged its expectations of evaluation quality. Although the Ministry shifted its initial advice against the use of Outcome Harvesting, its concern for bias remained.
A beneficiary of its Dialogue and Dissent policy framework, an Oxfam Novib-led NGO alliance, was among the first to roll out OH on a large scale for monitoring the results of policy advocacy with public and private sector actors, listening to their stories to capture outcomes. The alliance respected the principles underlying the methodology yet reduced the risk of such bias in designing its harvests: routinely asking for evidence and an analysis of external actors and factors’ contributions. Another risk mitigation measure was outsourcing substantiation to independent consultants applying an innovative, inclusive approach to contribution analysis in the final evaluation. This effort also fed into a broader discussion of the ministry’s criteria for assessing evaluation quality.
