Examining a Participatory Evaluation Approach that Centers BIPOC Teacher-Leaders’ Lived Experiences in a K-12 School Leadership Preparation Program

Abstract Information: The educator workforce is in transition. Three years after the emergence of a global pandemic, widespread racial and ethnic injustice, and unresolved systemic influences that affect the quality of teaching and learning for all educators and students, the education field is triaging ever-increasing shortages and strains at all workforce levels. Amid the losses observed across the teaching profession, growing the pipeline of talented and prepared teacher-leaders and principals is urgent. This paper explores using a participatory evaluation model to advance the process and outcomes evaluation of a K-12 school leadership preparation program for BIPOC teacher-leaders. We will describe formal and informal qualitative and quantitative methods that center participants’ experiences when evaluating the program against its enacted intentions and outcomes. Further, we will examine the value of a participatory approach involving multiple stakeholders’ lived experiences to build a narrative about the program’s impact on participants’ strengths, needs, and experiences.

Relevance Statement: Addressing diversity at the K-12 teacher and school leadership levels is a social and economic imperative, particularly for BIPOC educators. One approach to realizing a more racially diverse teacher pipeline supported by research is improving school leaders' racial diversity. Bartanen and Grissom (2019) found that "Black principals increase the probability that a newly hired teacher is Black by 5–7 percentage points." The same study showed that schools with Black leaders had a lower likelihood of turnover among Black teachers. Nationally, data on public school principals show that only 23 percent of principals are non-White (NCES, 2020). Further, while 82 percent of one large Midwest school district's students are Black and Latinx, only 63 percent of principals share these identities. These data suggest that the current principalship does not reflect students' racial and cultural identities.

This paper presentation focuses on a nascent K-12 school leadership preparation program designed by partnering national non-profit and university leaders in teacher and school leader preparation. This program provides opportunities for BIPOC teacher leaders in Midwest school districts to safely explore their identity, develop as leaders of color, and build a supportive professional network in a field that has historically perpetuated systems and structures to limit opportunities for advancement (Fenwick, 2022). Given these factors, the evaluators chose a participatory evaluation model that centered BIPOC teacher leaders' lived experiences inside and outside the program. Participatory models prioritize stakeholder collaboration, knowledge, and insights throughout the evaluation—instrument design, data collection, analysis, problem-solving, and decision-making (Cousins & Earl, 1992). As a result, the evaluation findings are more contextualized.

This paper will highlight the value of a participatory evaluation approach to building program capacity and a robust participant-centered evidence base to learn fast and make iterative program decisions, particularly for burgeoning social impact programs. Quite often, the field privileges quasi-experimental (e.g., propensity score matching) and traditional experimental (e.g., RCT) to judge the value of a program. However, fledging programs with small sample sizes may need other options. We argue that participants' narratives of the program's impact integrated with pre-post measures can provide a powerful means for judging the value and promise of such programs.

Therefore, we use the paper presentation to illustrate how the evaluators worked collaboratively with program staff and stakeholders to operationalize its logic model and design a mixed-methods evaluation system (i.e., processes and structures). First, we will walk AEA conference attendees through shifts in the evaluator's role as instructor and capacity-builder and the practical challenges and solutions to executing a complex non-experimental evaluation plan amid volatile conditions. Next, we will demonstrate how the evaluators partnered with program designers to develop cycles of data-driven action-oriented decision-making to ensure the program was responsive and adapted to participants' needs and experiences. Lastly, we will describe how we used a sequential method to shape evaluation measures based on what we learned from the stories BIPOC teacher-leaders shared during one-to-one conversations with program designers and evaluators.