Developing Evaluation Capacity Building of Social-Smotional Learning in Latin American Youth-Serving Organizations

Abstract Information: If building evaluation capacity in a single context is hard, can you imagine how hard it may be when you try to build capacity in multiple countries? In this session, we share lessons learned from a project that built the evaluation capacity of youth-serving organizations in Colombia and Peru. To add even more complexity, the initiative leaders (YMCA of Latin America and the Caribbean) were based in Mexico and the evaluation team (Hello Insight) was spread across Colombia, Spain, and the US. In spite of our contextual differences, we were able to center our work on how to promote young people’s social and emotional learning. This was possible thanks to investing enough time at the beginning of the initiative to create a shared theory of change, evaluation tool, and resources for implementation. This presentation will highlight 3 key areas of our work: 1) leveraging the voices of young people and frontline staff to build a shared theory of change across diverse programs and two countries, 2) learning about the attitudes and fears around evaluation of leaders and staff in diverse contexts, and 3) planning for the future of the initiative. Attendees will be encouraged to share their own experiences building capacity across diverse contexts.

Relevance Statement: Recently, the number of Youth-serving organizations that regularly have to evaluate their interventions has increased in order to improve their programs and be accountable for their efficacy and fundraising (Huffman et al., 2008; Suarez-Balcazar & Taylor-Ritzler, 2014). In the field of social emotional learning, the evaluation of these initiatives has been focused on public policies rather than in the evaluation of youth-serving organizations. These organizations in countries such as Colombia and Peru, have begun to build their evaluation capacity with external support. However, telling stories based on evaluation data is not easy. An organization needs multiple skills and backgrounds to sift through evaluation data, identify the most important findings, and craft a story that helps them improve and raise funds. These complexities are well documented in the Evaluation Capacity Building (ECB) literature (Preskill & Boyle, 2008; Huffman et al., 2008, Suarez-Balcazar & Taylor-Ritzler, 2014). The ideal situation is, especially in the beginning, to have the support of the evaluation experts, but using participatory activities gives them agency and feel engaged in exploring and understanding the data (Huffman et al., 2008; Clinton, 2014). Second, building evaluation capacity within a network could be more difficult considering their different cultures and context despite belonging to the same area and organization. This presentation contributes to the field by exploring the complexities to Evaluation Capacity Building in multi-country settings, and discussing a shared theory of change across diverse programs and two countries. We also show how leveraging the voices of young people and frontline staff was a foundational building block in working on the evaluation capacity of the different members of the initiative. Our presentation contributes to practice by sharing our work in the Latin American context. Through our method, program staff and youth have reflected on the collected data and used it for decision-making. Our process includes activities such as onboarding meetings, workshops to build a shared theory of change across programs, sessions to make meaning of the data, and sharing reflection guides to take action for enhancing the effectiveness of their programs. The audience could address similar challenges in cross-cultural contexts from our experiences and share their own. They could even reproduce and improve the procedure to build evaluation capacity across countries and organizations. They can also use some of these strategies to establish a conversation to exchange ideas and procedures about the challenges and achievements we have already faced. REFERENCES Clinton, J. (2014). The True Impact of Evaluation: Motivation for ECB. American Journal of Evaluation, 35(1), 120–127. https://doi.org/10.1177/1098214013499602 Huffman, D., Thomas, K., & Lawrenz, F. (2008). A Collaborative Immersion Approach to Evaluation Capacity Building. American Journal of Evaluation, 29(3), 358–368. https://doi.org/10.1177/1098214008318798 Preskill, H., & Boyle, S. (2008). A Multidisciplinary Model of Evaluation Capacity Building. American Journal of Evaluation, 29(4), 443–459. https://doi.org/10.1177/1098214008324182 Suarez-Balcazar, Y., & Taylor-Ritzler, T. (2014). Moving From Science to Practice in Evaluation Capacity Building. American Journal of Evaluation, 35(1), 95–99. https://doi.org/10.1177/1098214013499440