International and Cross Cultural Evaluation
Ashley Bishop, M.S. in Measurement & Evaluation
Technical Lead, Monitoring, Evaluation & Learning
Winrock International, United States
Drew Schmenner, M.A. International/Global Studies
Senior Technical Writer and Editor
School-to-School International, United States
Safal Khatri, M.A. Applied Developmental Psychology, M.Ed. Educational Assessment, Testing and Measurement
Technical Lead, Evaluation & Learning
Winrock International, United States
Location: White River Ballroom G
Abstract Information: The USAID Girls Leadership and Empowerment through Education (GLEE) Project is a 5-year contract set to conclude in September 2023, after reaching over 65,000 girls in the Kayes, Bandiagara and Douanza Regions in the Republic of Mali, located in West Africa. In 2023, Winrock International, the prime implementer, commissioned a final program evaluation of this project to EdIntersect, School-to-School International, and CERIPS. Following a similar design used on the external midterm evaluation, the design of the evaluation included quantitative data collection in reference to key program indicators and qualitative data collection through focus group discussions and key informant interviews. Winrock International’s Education Unit wanted to ensure that the stories of girls who worked with the project over the years were also captured through the final program evaluation in a meaningful way. To this end, the Most Significant Change methodology was selected to complement the mixed methods design to listen to girls in the program. After collecting these data, what story will the Winrock team be able to tell? How does the girls’ reporting of Most Significant Change in the life of the program complement the findings collected through more traditional mixed methods? The panel will be composed of multiple staff members who supported the evaluation process. Ashley Bishop, Technical Lead for Project M&E at Winrock, will facilitate the discussion and provide the overview of the evaluation process commissioned by Winrock International. Drew Schmenner, Senior Technical Writer and Editor from School-to-School International, will present how the evaluation team designed and carried out the Most Significant Change data collection process. Safal Khatri, Technical Lead for Evaluation & Learning, will also share how lessons were learned from the methodology to inform future evaluation designs.
Relevance Statement: This case study of an evaluation provides an interesting comparison of program evaluation results from two perspectives: 1) a more traditional mixed methods design, based on measurement planned via the original project framework, performance indicators, and learning questions, compared with 2) the self-identified changes highlighted by girls interviewed using Most Significant Change.
Presenter: Brenda Sinclair, PhD in Education – Winrock International
Presenter: Safal Khatri, M.A. Applied Developmental Psychology, M.Ed. Educational Assessment, Testing and Measurement – Winrock International
Presenter: Abou Coulibaly, n/a – Winrock International
Presenter: Ashley Bishop, M.S. in Measurement & Evaluation – Winrock International
Presenter: Drew Schmenner, M.A. International/Global Studies – School-to-School International