Theories of Evaluation
Thomas Schwandt, PhD
Professor Emeritus
University of Illinois at Urbana Champaign
Bloomington, Indiana, United States
Location: Room 301
Abstract Information: In our recent book, Evaluating and Valuing in Social Research (Schwandt & Gates, 2021), we introduced a new way of thinking about the meaning of evaluating. It is common to assume that evaluating means determining the value of something or assigning value to it, where value means merit, worth, importance, or significance. That something could be an action, a strategy, a product, program, policy, and so on. For example, a real estate appraiser assigns a market value to a property; an experimentalist determines the effectiveness and efficiency of a social intervention compared to some alternative; a manager decides the value of an employee’s contribution to an organization by conducting a performance assessment. In each of these examples, evaluating means settling or establishing value. Evaluating, as thus portrayed, is a typical undertaking found in all aspects of everyday life. An alternative way of understanding evaluating introduced in our book regards it as a dynamic, contested, and fluid undertaking more concerned with developing value than determining it. It involves challenging, expanding, and pluralizing assessments of value as parties to an evaluation learn and act. In considering the difference between these two ways of understanding evaluating, we call to mind that the notion of determining is associated with acts of establishing, concluding, and settling on something, while the notion of developing is associated with acts of creating, cultivating, elaborating, and working out something. In this talk, we will expand upon and illustrate this idea, its justification, and implications for practice.
Relevance Statement: There is a school of thought that argues that we live within ongoing stories that we must constantly tell ourselves as a condition of being able to perform intentional actions such as being a professional of a particular kind. There is a longstanding story that evaluators tell themselves and others about what it is that they do and why. The story is told something like this: “Professional evaluators are in the business of determining the value of actions, strategies, products, programs, policies, and so on. Working as independent experts largely uninvested in what is being evaluated, we employ (at least in principle) a four-step logic of evaluation (select criteria of merit > set standards of performance on criteria > gather data on performance > synthesize results into value judgment) to determine the merit, worth, significance, or importance of whatever it is we are evaluating. Our work on the determination of value is, in turn, a means to answer some rather straightforward questions about what was accomplished; for example, how did we do? Did we achieve what we set out to achieve? Can we attribute observed outcomes to what was done?” In the past decade or so, a different, yet not quite coherent, new narrative of evaluating is emerging as the field of evaluation wrestles with the utility of its longstanding narrative for addressing intersecting, complex, wicked problems that we face in socio-political-environmental world. Evaluators are not only exploring new means for conducting evaluating but new ideas of what the practice itself should be about. Evaluation scholars and practitioners are borrowing ideas from the fields of systems thinking and complexity science; looking to insights from public policy analysis and public administration to explore concepts of co-creation and coproduction; experimenting with how evaluating might reflect greater attention to thinking and working politically; attempting to recentering ethics in evaluating as something more than a matter of acting professionally; investigating how to refocus evaluating on agency and practice by focusing on questions such as “what should be done?” and “where do we go from here?” All of this is taking place in an intellectual and political environment highly attuned to a complicated set of interrelated issues focused on achieving diversity, equity, inclusion, and social justice across all sectors of society. A key characteristic of this wide-ranging discussion is that it is highly interdisciplinary in nature, moving outside of the normal confines of the evaluation method-and-models literature and focused on transformational change in the definition of the practice of evaluating. This expert lecture will help give some shape to this emergent narrative by first naming it as a story about evaluating as developing value, and then examining the assumptions that underlie this way of thinking and acting like an evaluator, contrasting these assumptions with those related to the evaluating as the determination of value. Listeners to the lecture might consider this an exercise in ‘re-storying’ evaluation practice that invites rethinking evaluators’ professional role and responsibilities.