A Story of the Future of Theory-Driven Evaluations: From Better Theory Visualizations to Artificial Intelligence

Abstract Information: This Expert Lecture will first summarize the powerful of story of how the practice of theory-driven evaluation has developed over time. Based on this foundational story, some of the newest cutting edge innovations for developing and using evidence-based program theories to improve the design and evaluation of programs will be presented and discussed. A special emphasis will be placed on exploring the wide range of options for visualizing complex program theories, and how recent developments in artificial intelligence can enhance the participatory process of developing program theories with primary and secondary stakeholders. Finally, Professor Donaldson will summarize his vision for a better future for theory-driven evaluation practice that has been presented in his new book “Introduction to Theory-Driven Program Evaluation: Culturally Responsive and Strengths-focused Applications (2022).” Huey T. Chen, Jessica Renger, and Charles Gasper (session discussants) will critique and discuss the expert lecture, and the audience will be allotted 10-15 minutes to discuss Professor Donaldson’s vision for the future of evaluation practice and be invited to provide their own ideas and perspectives.

Relevance Statement: Huey T. Chen published the first textbook on Theory-Driven Evaluations in 1990. In 2007 Professor Donaldson in his book on program theory-driven evaluation science summarized the conceptual and methodological advances of theory-driven evaluation since 1990, and offered an innovative definition and vision for evaluation practice that has been implemented well in the past 15 years. He defined program theory-driven evaluation science as “the systematic use of substantive knowledge about the phenomena under investigation and scientific methods to improve, to produce knowledge and feedback about, and to determine the merit, worth, and significance of evaluands such as social, educational, health, community, and organizational programs.” More recently, in an American Journal of Evaluation article, Patton (2018) reviewed a variety perspectives on evaluation as a transdiscipline and as a science, and provided a broader definition that “evaluation science is systematic inquiry into how, and how well, interventions aimed at changing the world work.” One of the cries heard often around the world during the recent pandemic was ‘follow the science.” The relevance of that cry for theory-driven evaluation science and practice today and tomorrow is explored in depth in Professor Donaldson’s new book “Introduction to Theory-Driven Program Evaluation: Culturally Responsive and Strengths-focused Applications (2022).” This expert lecture will summarize and share that work within the framework of the 2023 AEA Theme: The Power of Story, the AEA Culturally Competency Statement, Guiding Principles, the AEA Evaluator Competencies, and the latest advances in visualizing evidence-based program theories and artificial intelligence (AI). One of the core features and foundations of the transdiscipline of evaluation is the first AEA principle systematic inquiry. Systemic inquiry of the highest standard is a primary goal of theory-driven evaluation practice. The new AEA evaluator competencies and the AEA principle on competence will be used to illustrate what evaluators need to develop and master to do this work well. The principles of integrity, respect for people, the common good and equity, and cultural competence will be illustrated in a variety of culturally responsive and strengths-based applications. This Expect Lecture will be link closely to the AEA principles and the other unique knowledge that has accumulated at the center of the transdiscipline over the past 35 years.