Using Participatory Evaluation to Understand the Implementation and Impact of a Statewide Kindergarten Transition Program

Abstract Information: Using a participatory evaluation approach, a team including university researchers, stakeholders from a state educational organization, kindergarten teachers, and parents/caregivers joined together to identify critical implementation factors as well as the effectiveness of a statewide kindergarten transition program. During the mixed methods evaluation, the team learned about a multitude of factors that impact program implementation from teacher and parent/caregiver perspectives including location of sessions, length of sessions, content of sessions, language spoken during sessions, and family engagement during sessions. The team used a variety of evaluation tools, including teacher and parent/caregiver surveys, assessments in multiple domains of development, document review, and interviews, to inform the evaluation. In addition, progress toward goals and benefits of the program were uncovered via the multiple perspectives involved in the evaluation. Over a two-year evaluation period, feedback from teachers and parents/caregivers helped refine evaluation strategies and instruments and expand the goals of the program. Finally, the team realized the significance of these voices in informing statewide transition to kindergarten efforts as well as other educational transitions that occur in young children’s lives. Join us as we discuss the power of participatory evaluation to inform program evaluation and highlight the multiple stories of success and challenge that emerge when elevating ideas and perceptions of program participants.

Relevance Statement: Participatory approaches to research are becoming increasingly common across a variety of sectors including action research and evaluation (Mertens, 2020). A participatory approach allows evaluators to understand complexities and contexts that are integrated within programs, enables ideas from differing perspectives to emerge, and facilitates relationships to increase trust and promote program improvement. Findings from participatory evaluations often are more likely to uncover unforeseen benefits and challenges as well as identify contextual strategies to enhance benefits and reduce challenges. Researchers from a state university partnered with implementers of a kindergarten transition program to evaluate the implementation and effectiveness of the program. The kindergarten transition program serves approximately 1,417 children and their families in 26 counties across the state each year. Through this partnership, a representative sample of 16 kindergarten teachers implementing the program and parents/caregivers in their classrooms were also engaged in shaping the evaluation. During a two-year period that included bi-weekly meetings with program implementers and evaluators, a mixed methods evaluation that included document review, interviews, surveys, and assessments was used. In Year 1, 16 teachers worked with the team and 10 parent participants were interviewed. This shaped the Year 2 evaluation that expanded the evaluation design and included 30 teachers and more than 350 parents within their classrooms. The program was designed to ease the transition to kindergarten for children and their families, but the participatory evaluation also identified program benefits such as teachers gaining useful insights into their incoming kindergarten class that shaped instruction. In addition, the parents/caregivers appreciated the connection with the teachers and the activities modeled during the sessions but noted their uncertainties about the meaning of school readiness for their current rising kindergarteners as well as their younger siblings. Through the evaluation, the location of the sessions (school, community, or home) did not appear to be a factor in defining the quality of the sessions; however, the teachers' engagement and relationship-building skills were relevant. Teachers also identified the need to adapt the curriculum that shapes the six sessions based on the engagement and appropriateness for the family and child. Some families are balancing multiple children and competing priorities during the sessions, but some stories from the teachers and parents identified the benefits of engaging older and younger children in the sessions as well. If the evaluation had used traditional methods to evaluate the effectiveness of the program on its goals only, the partnership would have missed key details that drive success and impacts beyond the goals. Evaluators will share how inclusive and context-based understandings (critical evaluator competencies) shaped the methods and findings. State educational organizational representatives will discuss how the participatory approach broadened thinking about implementation and goals. Both groups will share how the evaluation findings are being used to transform kindergarten transition efforts statewide, expand the program to prekindergarten, and engage teachers and parents/caregivers in evaluation, program implementation, and policy development. Mertens, D. M. (2020). Research and evaluation in education and psychology: Integrating diversity within quantitative, qualitative, and mixed methods (5th ed.). SAGE.