Improving quality of training based on evidence from evaluation – lessons learned from Kosovo and Ethiopia

Abstract Information: Program evaluations help understand the impact of the program and are necessary to ensure the programs are meeting their goals and objectives in the most effective way possible. Ongoing concurrent evaluations or those with baseline, midline and end-line studies help improve the quality of the program itself. We draw upon our experience from two separate independent evaluations of training programs, conducted in disparate parts of the world, to tell the story of how evaluations conducted at three points over the life of the program – beginning, mid-point and end of the program – led to stronger and more robust implementation and improved the quality of training as well as the impact of the program. These evaluations drew upon quantitative data collected through surveys and review of records to critically narrate the experience of participants impacted by the program, and qualitative data collected through interviews with participants, supervisors and program implementers to fill in the details of program implementation and complete the story of the impact on the participants. In addition to a discussion of the methodology used, we talk of the lessons learned from the conduct of these two long-term evaluations and the potential applicability of the approach in a range of country, program and population settings.

Relevance Statement: The theme of this conference is story telling – specifically, how storytelling contributes to and shapes the narrative of evaluations, and how often-times the story leads to an evaluation. The proposal described above is based on two mixed-methods studies conducted in different parts of the world that were extremely well-received both by the client and by the implementers of the program. These initiatives suggest that evaluations can be collaborative and used to strengthen a program rather than to just point out the problems. In addition, these evaluations point to the need for both quantitative and qualitative data to construct the full picture of the impact of the training program on the participants. Every voice counts – and listening to a range of participants helps the evaluators construct a deeper and better-nuanced picture of the whole and make usable recommendations for the implementers and those funding the program