Program Design
.jpg "Kathleen Dean, PhD photo")
Kathleen Dean, PhD
Senior Research and Evaluation Associate
The Rucks Group, United States
Lyssa Becho, Ph.D.
Principal Research Consultant
Western Michigan University, United States
Location: Room 209
Abstract Information: Appreciative Inquiry (AI) starts with individuals sharing stories with one another to elicit a shared understanding that will help to co-create an aspirational future. Using AI as an approach to evaluation generates a wealth of information that tells the story of organizational strengths, opportunities, and goals to create a shared vision. EvaluATE, the evaluation hub for the National Science Foundation’s Advanced Technological Education program, partnered with their external evaluator (The Rucks Group, LLC) to design a strategic thinking and planning process using AI to engage participants, generate data, and create a future vision to inform subsequent program design. During this demonstration, presenters will guide attendees through applying an AI lens – from developing questions that generate stories to imagining organizational futures – and the tools used to accomplish this virtually. The demonstration will conclude by sharing lessons learned and recommendations for attendees to use this approach in their own practice.
Relevance Statement: The 2023 conference theme asks us to consider the power of story in evaluation. As evaluators, we help others to define and answer key questions, generate insights, and identify actions to inform their next steps in program design and improvement. Appreciative Inquiry (AI) is an approach to evaluation that begins with stories. These stories can build relationships among participants, provide rich descriptions of organizational context, and generate actionable data. Using AI as a framework for program design provides valuable information that tells the story of an organization’s strengths, opportunities, and aspirations resulting in an actionable, shared vision. This demonstration will provide an overview of using AI in evaluation for program design and guide participants through the process of how AI was used to facilitate a collaborative strategic thinking and planning process for a grant-funded center. The idea to use an AI approach was brought to EvaluATE, the evaluation hub for the National Science Foundation’s Advanced Technological Education (ATE) grant program, by their external evaluators, The Rucks Group, LLC. The EvaluATE team had expressed a need to consider their strategic direction as they continue to grow. AI is well-suited as a framework for strategic thinking and planning because it emphasizes a high level of organizational engagement, co-construction, a focus on strengths, and a positive future. A core principle of AI is that we can understand a lot from the stories that people share prompted by important questions. Stories have power because they create images and are easy to remember. Based on the “4-I” appreciative inquiry framework – Inquiry, Imagine, Innovate, Implement – adapted by Preskill and Grindle (https://www.fsg.org/tools-and-resources/guide-appreciative-inquiry) , the EvaluATE project staff engaged in a three-month, virtual process using stories to discover what has made excellence possible in the organization and to explore what their organization could be. During this demonstration, participants will learn the basics of AI, how it was used to achieve EvaluATE’s strategic thinking and program design goals, and how these methods and tools can be applied to their own work. The presenters will demonstrate the important role that stories play throughout the AI process in ways that generate data and inform program design. The demonstration will conclude by sharing the lessons learned and recommendations for attendees in using this approach in their own practice. The presenters are the project director for EvaluATE and their external facilitator from The Rucks Group. EvaluATE strengthens evaluation capacity within the NSF ATE community through virtual and in-person trainings, technical assistance, peer-to-peer learning, and resource development. Since 2008, EvaluATE has developed over 55 webinars, 15 workshops, and 35 evaluation resources. The Rucks Group is a research and evaluation firm that engages with clients to make data-informed decisions. Founded in 2008, TRG works with institutions of higher education, non-profits, and foundations with a specific emphasis on STEM and technical education and workforce development.