34 - Transformative Evaluability Assessment: Engaging Local, Cultural, and Contextual Knowledge to Understand a Program’s Story

Abstract Information: The purpose of this workshop is to explore and engage the transformative use and potential of evaluability assessment (EA). EA was developed in the 1970s as a pre-evaluation activity for determining if a program was ready for outcome evaluation, with a focus on managers as primary intended users. Much like evaluation in general, EA theory and practice have evolved to address the complex needs of programs and their communities. No longer tied exclusively to management decisions about outcome evaluation, EA can be used as a collaborative evaluation approach at any point in a program’s lifecycle. Transforming our understanding and application of EA unlocks its potential to engage program and organization communities in evaluation—incorporating local, cultural, and contextual knowledge to understand the story of a program through diverse perspectives, experiences, and values. A transformative EA approach can address program complexity, support culturally responsive and equity-focused evaluation, and build evaluation capacity. Using our four-component EA model as a framework, and through examples, case scenarios, small group activities, and discussion; workshop participants will consider and apply a transformative EA approach to engage local, cultural, and contextual knowledge.

Relevance Statement: The transformative paradigm is a view and frame for inquiry that centers questions of power and privilege, understanding multiple socially constructed realities, shared dialogic space, collaborative evaluation learning, and social justice (Creswell, 2018; Mertens, 2008). As evidenced by published EA work, some evaluators are embracing and innovating EA as a transformative approach, while others limit their conceptualization and use of EA to its traditional, 1970s form. Although traditional EA remains useful as a pre-evaluation activity to determine a program’s readiness for outcome evaluation, ways of thinking about and using EA have evolved over the decades. As early as the 1980s EA was being used as a stakeholder involvement approach to evaluation, as formative evaluation (Smith, 1989), and to develop a quality plan for subsequent evaluation (Rutman, 1980). In the early 2000s, Thurston and Potvin (2003) advocated ongoing participatory EA of social change programs. More recently, EA has been used for program development (Lam & Skinner, 2021), culturally-responsive evaluation (Clarke et al., 2021), supporting social justice (Wretman et al., 2021), gender-responsive evaluation (United Nations Women, 2015), and integrating evaluation approaches (Deane et al., 2019). Further, EA can facilitate stakeholder engagement; deep understanding of a program and its context; evaluation capacity building, including evaluative thinking and improved use of evaluation (see Lam & Skinner, 2021); and development of a learning organization (United Nations Women, 2015). These purposes and outcomes of EA align with the transformative inquiry paradigm. Using EA in this way centers local, cultural, and contextual knowledge, providing an understanding of a program’s story through diverse perspectives, experiences, and values.