Theories of Evaluation
John Lavelle, PhD
Assistant Professor
University of Minnesota, United States
Sebastian Lemire, PhD
Senior Associate
Abt Associates, United States
Sebastian Lemire, PhD
Senior Associate
Abt Associates, United States
Katrina Bledsoe, PhD
Principal Associate
Abt Associates, United States
Melvin Mark, PhD
Professor Emeritus of Psychology
Penn State University, United States
Melvin Mark, PhD
Professor Emeritus of Psychology
Penn State University, United States
Location: Room 201
Abstract Information: As part of his Presidential Address at the 1998 American Evaluation Association conference, Will Shadish famously declared that evaluation theory is who we are. The historical context for his remark was, of course, that evaluation as a transdisciplinary field was coming of age. Accordingly, there was a collective need and interest in defining what evaluation is and should be—to draw boundaries around the practice and profession of evaluation. Over the years, and as the range of evaluation approaches and methods continued to grow, evaluation scholars have depicted the growing landscape of evaluation in the form of a tree (Christie & Alkin, 2004), a periodic table (Vaca, 2018), and a metro map (Lemire, 2020), among others (Mertens & Wilson, 2019). The purpose of this session is to explore the different visualizations that have been used to map the evaluation landscape and to discuss their benefits and implications for the practice and profession of evaluation.
Relevance Statement: Speaking directly to the important role and purpose of evaluation theory, Shadish, Cook, and Leviton—in their classic volume on evaluation theory—define theory as a “body of knowledge that organizes, categories, describes, predicts, explains, and otherwise aids in understanding and controlling a topic”. As Shadish went on to discuss, evaluation theory matters for the practice and profession of evaluation in several important ways: by providing a common language (e.g., “formative” vs. “summative”, by identifying important issues (e.g., evaluation use, nature of causation, equity), by defining evaluation from other professions (e.g., education, economics), and by establishing a unique knowledge base (what makes “evaluators” unique). Over the years, the range of evaluation theories and approaches, designs, and methods, has grown substantially. The evaluation landscape has and continues to grow—expanding yet maintaining the boundaries of what evaluation is and can be. As a way of organizing this expanding landscape, evaluation scholars and practitioners have over the years developed different visualizations of evaluation theory—invoking imagery of trees, periodic tables, and metro maps. These different visualizations are important for several reasons. In line with the conference theme, these visuals can be viewed as different ways of telling the story of evaluation. In each their own way, the different visuals provide a narrative—a way of making sense—of the evaluation world used in day-to-day evaluation practices. In this way, the visuals also reflect the authors’ background, relationships, and experiences in evaluation. Reaching beyond individual evaluators, the visuals also hold potential implications for the profession of evaluation. By mapping the evaluation landscape, the visuals signal what evaluation is and could be—establishing boundaries between evaluation and other professions. Informed by Shadish, Cook, and Leviton, the visuals can be viewed as different ways of categorizing, describing, and controlling the topic of evaluation. With these reflections as our backdrop, the purpose of the panel is both to provide an overview of the different visuals—and narratives—of evaluation theory and to discuss the implications of these visuals for the practice and profession of evaluation. What are the implications of depicting evaluation theory in different ways? What are the stories we are telling ourselves about evaluation—about who we are and what we can be? Whose stories are not being told? These are some of the questions we will discuss during the session. The session aligns with the AEA Evaluator Competencies on professional practice, focusing on what makes evaluators distinct as practicing professionals as grounded in AEA’s foundational documents and guidelines.
Presenter: Katrina L. Bledsoe, PhD – Abt Associates
Presenter: Sebastian Lemire, PhD – Abt Associates
Presenter: John Lavelle, PhD – University of Minnesota
Presenter: Melvin M. Mark, PhD – Penn State University