Integrating Technology into Evaluation
Bianca Montrosse-Moorhead, PhD
Associate Professor
University of Connecticut, Connecticut, United States
Sarah Mason, PhD
Director, Center for Research Evaluation
The University of Mississippi, United States
Nina Sabarre, PhD (she/her/hers)
Founder & CEO
Intention 2 Impact
Arlington, Virginia, United States
Kathleen Doll, PhD (she/her/hers)
Partner & COO
Intention 2 Impact
Irvine, California, United States
Sahiti Bhaskara, MPH, BDS
Senior Evaluation Consultant
Intention 2 Impact, United States
Linda Raftree, BA in Anthropology
Founder
MERL Tech, New York, United States
Aileen Reid, PhD (she/her/hers)
Assistant Professor
UNC Greensboro
Greensboro, North Carolina, United States
Zach Tilton, n/a
Evaluation Consultant
Western Michigan University, United States
Location: Room 103
Abstract Information: Recent groundbreaking advances in artificial intelligence (AI), namely the November 2022 launch of ChapGPT, have ushered in a renewed wave of intrigue and discourse surrounding the individual, collective, and global implications of AI. To date, evaluations of and with artificial intelligence have largely been underdeveloped and under-explored. In this session, four sets of panelists will share their use of AI in their evaluation practice, offer implications of artificial intelligence in conducting and the teaching of evaluation, and invite discussion on questions such as: What role should artificial intelligence play in evaluation? What does it make possible for smaller evaluation firms? What practical, ethical, methodological, and philosophical challenges does emerging AI pose? Which parts of evaluation are inherently human? How much will evaluators and evaluation users trust the products of AI-generated or AI-assisted work? How do we train emerging evaluators to work in a world in which AI is prevalent?
Relevance Statement: While the invention of artificial intelligence (AI) technology and tools have been on the rise in recent decades, the November 2022 launch of ChatGPT, a chatbot that uses generative AI, has ushered in a renewed wave of intrigue and discourse surrounding the future of AI. Generative AI is distinct from earlier AI models in that it can create entirely new content. Generative AI is intended to create new text, images, audio and video content using its training data as a foundation for creation. ChatGPT’s natural language processing also enables easy human-to-computer communication, which facilitates rapid and complex conversations that are fundamentally different to earlier generations of artificial intelligence. This groundbreaking advance has spurred disciplines across the globe to grapple with the individual, collective, and global implications of AI. To date, evaluations of and with artificial intelligence have largely been underdeveloped and under-explored within the field of evaluation. In this session, four sets of panelists will share their use of AI in their evaluation practice, offer implications for artificial intelligence in conducting and the teaching of evaluation, and invite discussion on the questions: What role should artificial intelligence play in evaluation? What does it make possible for smaller evaluation firms? What practical, ethical, methodological, and philosophical challenges does emerging AI pose? Which parts of evaluation are inherently human? How much will evaluators and evaluation users trust the products of AI-generated or AI-assisted work? How do we train emerging evaluators to work in a world in which AI is prevalent?
Presenter: Aileen M. Reid, PhD (she/her/hers) – UNC Greensboro
Presenter: Zach Tilton, n/a – Western Michigan University
Presenter: Bianca Montrosse-Moorhead, PhD – University of Connecticut
Presenter: Sarah Mason, PhD – The University of Mississippi
Presenter: Nina Sabarre, PhD (she/her/hers) – Intention 2 Impact
Presenter: Kathleen Doll, PhD (she/her/hers) – Intention 2 Impact
Presenter: Sahiti Bhaskara, MPH, BDS – Intention 2 Impact
Presenter: Linda Raftree, BA in Anthropology – MERL Tech