Internal Evaluation
.jpg "Lenka Berkowitz, M.S. (she/her/hers) photo")
Lenka Berkowitz, M.S. (she/her/hers)
Evaluation Manager
Alliance for a Healthier Generation
Philidelphia, Pennsylvania, United States
Elena Kuo, PhD (she/her/hers)
Sr. Evaluation and Learning Consultant
Center for Community Health and Evaluation, Kaiser Permanente Washington Health Research Institute
Seattle, Washington, United States
Location: White River Ballroom A
Abstract Information: The success of an evaluation hinges on an evaluator's ability to help others make sense of the findings, which is crucial for developing a strong program narrative. In this skills-building workshop we will explore an innovative approach to engaging program staff in reflecting on evaluation findings to co-create a compelling program story. We will use the example of work conducted by the Alliance for a Healthier Generation (Healthier Generation) in partnership with Kaiser Permanente to advance equitable whole-child health. Through this partnership, Healthier Generation works with school districts in 8 regions across the U.S. helping to make policy, practice, and environmental changes. We will demonstrate how evaluators can harness program staff expertise to collaboratively create a narrative that accurately and authentically captures the program's impact using our recent evaluation work. The workshop will spotlight “meaning-making sessions,” a guided process that facilitates group reflection to uncover new insights from data and generate actionable next steps for program improvement. The theoretical grounding combines a utilization-focused approach to evaluation with feedback loops and reflective practice. Attendees will have the opportunity to learn from our recent experiences with meaning-making sessions conducted as part of the program evaluation and will leave the session with practical steps for implementing this approach in their own evaluation work.
Relevance Statement: Together with Thriving Schools in all eight Kaiser Permanente markets, Healthier Generation is convening and equipping health, school, and community champions with an array of tools and resources that prepare them to equitably transform environments surrounding young people and their families. As part of a comprehensive evaluation plan, Healthier Generation utilizes meaning-making sessions as one component of the evaluation process to guide program improvement efforts. By utilizing reflective practice, program staff can play an active role in interpreting data and designing action steps, leading to a more accurate and inclusive understanding of program impact. In this workshop, presenters will show the value of meaning-making by drawing on experiences using this guided process in a recent evaluation of a multisite intervention focusing on policy, practice, and environmental changes that aim to improve the health and well-being of students, staff, and families. The workshop emphasizes the importance of justice, equity, diversity, and inclusion principles in evaluation work. The meaning-making session process is important for elevating participant voices – and has broader applications as it could be implemented with communities that evaluators work in or with organizational leadership or the board of directors. This approach is particularly relevant because it recognizes the importance of involving stakeholders in the evaluation process to tell their stories more authentically. Additionally, it rejects the traditional top-down evaluation approach where evaluation is done “to” program participants and instead reflects the field’s current emphasis on participatory and collaborative approaches where evaluation is done “with” program participants. Including program staff in shaping the narrative of the program’s success and challenges is an important contribution to this shift. The approach combines the utilization-focused evaluation as defined by Michael Quinn Patton with the work on feedback loops and reflective practice in evaluation. The presenters will share the unique framework that they developed combining these approaches and thus contributing to the emerging new trends in evaluation theory. There is great value to the attendees as the workshop will not only describe the meaning-making process but also provide practical skills and tips on implementing them. Internal evaluators are facing unique challenges in terms of implementing participatory evaluation methods. Yet, including staff in all stages of evaluation has many benefits including building staff’s skills, ensuring relevant and actionable data collection methods, and increasing buy-in and motivation for evaluation activities. This can help program staff see their work reflected in the data which in turn may lead to better program performance. By focusing on the complexities of implementing reflective practice from the perspective of internal evaluators, the workshop is also contributing to the limited knowledge base on internal evaluation. Below is a tentative agenda outline for the workshop: - Introduce the concept of meaning-making sessions, the theoretical framework, and guiding principles - Demonstrate the value and challenges of this approach using an example of a recent Healthier Generation evaluation - Reflect on how meaning-making can be incorporated into participants’ current evaluation work - Share a one-pager with practical tips on how to implement meaning-making sessions