Evaluation Managers and Supervisors
Melissa Chapman Haynes, PhD
Director of Evaluation
Professional Data Analysts, United States
Kate LaVelle, PhD
Principal Evaluator
Professional Data Analysts, United States
Location: Room 102
Abstract Information: Evaluation management may initially seem one of the least likely places to interweave storytelling; however, in this complex, constantly changing, and very human space of management learning through stories of evaluation management is an excellent way to make visible an often-invisible aspect of our work. A 2009 New Directions in Evaluation (NDE) Issue (#121) focuses on Managing Program Evaluation. Four studies were shared, as well as a typology of managing evaluation. This typology was organized into the following categories: ethos, stance, gaze (i.e., minimum required knowledge in managing evaluation). In this think tank we will first review the typology in the 2009 NDE issue and share some stories from the authors on their combined forty-year experience in doing and managing evaluations and evaluation teams. The 2009 NDE issue drew from the management literature, the psychological development of expertise, AEA guiding documents, as well as professional practice and experience. In this session we would like to ground our work in this publication and then focus the discussion on the omission of storytelling and narrative about evaluation management as a way to build this professional competency (AEA competency 4, Planning and Management). “Stories are memory aids, instruction manuals and moral compasses,” according to Aleks Krotoski (http://alekskrotoski.com/). The guiding issue to be addressed is how to incorporate stories of evaluation management into a proposed update to the 2009 NDE issue: how can we engage, inform, and/or inspire through storytelling focused on evaluation management? How can we advance this competency through storytelling?
Relevance Statement: With the support of competencies by AEA, as well as the development of a CHC self-assessment and other guiding documents, taking a new lens on evaluation management is essential. This is a critical but often invisible aspects of evaluation – it’s clear when it’s not in place but when it’s done well is often unrecognized. Why tell stories around this critical issue? Psychological studies have found that people forget more than 40% of information told to them within just 24 hours (Murre & Dros, 2015); emotional memories influences learning and memory (Tyng et al., 2017). How can we lean into storytelling as a way to build the literature on evaluation management? Management is a critical component of the AEA evaluator competencies as well as the Centers for Disease Control and Prevention self-assessment. Here are the competencies in the CDC self-assessment, Managing Evaluation section: • Budgeting • Identifying the appropriate mix of skills needed for a specific evaluation (e.g., interpersonal, technical) • Hiring or contracting for evaluators • Establishing feasible scope and timeline • Establishing and using systems to document evaluation implementation (e.g., documenting processes, procedures, and decisions) • Monitoring and communicating progress to stakeholders • Addressing ethical issues as they arise • Addressing political issues that arise • Planning for issues of use with stakeholders • Evaluating your evaluation (meta-evaluation) The content from this session will be used to inform a proposal to develop a follow-up proposal to the 2009 NDE publication. References Murre, J.M.J., & Dros, J. (2015). Replication and Analysis of Ebbinghaus’ Forgetting Curve. Plos One, https://doi.org/10.1371/journal.pone.0120644 Tyng CM, Amin HU, Saad MNM, Malik AS. The Influences of Emotion on Learning and Memory. Front Psychol. 2017 Aug 24;8:1454. doi: 10.3389/fpsyg.2017.01454. PMID: 28883804; PMCID: PMC5573739. US Chamber of Commerce. Elevate your Effectiveness as a Leader with Storytelling. https://www.uschamber.com/co/grow/thrive/storytelling-leadership-tool#:~:text=Storytelling%20in%20leadership%20plays%20a,employees%20in%20the%20right%20direction.