Roadmap for conducting a meta-evaluation: approach and insights.

Abstract Information: Meta-evaluations evaluate (other) evaluations with the overarching goal of gaining insights into their quality. In the end, meta-evaluations are used to identify strengths and weaknesses of evaluations, improve evaluation practice, and strengthen accountability for stakeholders. This short presentation aims to describe how to conduct a meta-evaluation from the perspective of an evaluator. It is based on the experiences of a recently completed meta-evaluation of the German Institute for Development Evaluation, in which eleven organizations with 296 evaluations were involved. The story includes the following subtopics on 20 slides. The presentation 1. introduces the concept of a meta-evaluation, 2. presents criteria that are helpful in selecting relevant organizations and evaluations, 3. shows how to identify the relevant quality standards that define evaluation quality, 4. illustrates how to develop an analysis grid, 5. describes the process of analyzing a variety of evaluation and organizational documents, 6. presents findings from the published cross-organizational meta-evaluation, 7. provides an overview of opportunities, challenges, and lessons learned. For each sub-topic one to four slides are used to presents the content.

Relevance Statement: Meta-evaluations are an important instrument to ensure that evaluations are of high quality. In the policy field of international development cooperation, there is a growing number of meta-evaluations. The instrument “meta-evaluation” can be used for different purposes and are therefore important on different levels. Single organizations can (have) meta-evaluations carried out in order to account for the evaluations of their interventions for which they are responsible, as well as to learn from them and use these findings for the further development of their evaluation work and organizational strategy. Research institutions, on the other hand, can conduct cross-organizational meta-evaluations to generate evidence for strategic decisions in the policy field under study or to present possible further developments of the instrument for the future.