Evaluative Thinking for Transformative Change: Building Capacity in Early Care and Education Systems

Abstract Information: A challenge for evaluators engaging in systems change work is for interested parties to share and articulate understandings about what change will look like when it happens – how it can be observed and measured – and how their stories can be told. Participating in a statewide early childhood systems change initiative, and drawing on a relational systems approach, evaluators gathered diverse statewide partners to collaborate and create shared understanding and tools around equitable systems evaluation. In interviews, key informants (system leaders) articulated features of evaluative thinking, which varied over time and by leadership role in the system. System leaders were interviewed at two timepoints, with a focus on understanding their perspectives of system shifts related strategic plan goals of quality, access, collaboration, and alignment. Evaluative thinking at all levels was observed, with variation across types of thinking and complexity. 

Relevance Statement: Quality Care and Education for children birth-five years old has been identified as a key area of prevention, known to enhance well-being for children, families and communities. However, the early childhood mixed delivery system is constructed of services that are not easily accessible, affordable, or aligned for families. The Preschool Development Grant Birth to Five (PDG B-5) funded states to enhance state systems for early care and education programs and services. Grounded in a developmental systems approach (Lerner, 2006), one state performance evaluation team developed an evaluation network team of diverse community program and evaluation partners. They engaged in a systemic community of practice to build shared understanding around systems (Midgley, 2003) and build capacity to develop and evaluate programs using an equitable systems lens. A challenge in engaging in systems change is for interested parties to share and articulate understandings about what change will look like when it happens – how it can be observed and measured – and how those stories can be shared. Participating in a statewide early childhood systems change initiative, and drawing on a relational systems approach, evaluators gathered diverse statewide partners to collaborate and create shared understanding and tools around equitable systems evaluation, as part of the “state story”. A relational systems evaluation approach was adopted to investigate how leaders were using evaluative thinking (Urban, et al. 2021) to describe aspects of change in the system. Critical thinking applied in the context of evaluation, evaluative thinking is motivated by curiosity, an interest in problem-solving, and placing value on evidence. Evaluative thinking involves identifying assumptions, pursuing understanding through perspective taking, posing thoughtful questions, and making informed decisions. System key informants and individual project leads were interviewed three times from 2019 – 2023 regarding their perspectives of the existing early childhood mixed delivery system, and how funded efforts were aligned with the newly developed state strategic plan. Researchers examined how leaders employed evaluative thinking in their responses. Using content analysis coding they noted how leaders 1) identified assumptions, such as how and why a program worked; 2) used perspective-taking, by leveraging the perspectives of others; 3) asked thoughtful questions, meant to enhance the program; and 4) discussed making informed decisions by relying on evidence and planning. Evaluative thinking aspects were refined in response to the specific systems work. Implications for ongoing efforts include engaging with interviewed participants in explicit conversations about evaluative thinking and its value for telling stories of success and learning, and exploring ways that evaluative thinking can be brought to program planning and implementation earlier in the process.