Internal Evaluation
.jpg "Benjamin Williamson, EdM (he/him/his) photo")
Benjamin Williamson, EdM (he/him/his)
Research and Evaluation Scientist
CYBER.ORG
Bossier City, Louisiana, United States
Alicia Kiremire, PMP (she/her/hers)
Owner
FlowStream Management LLC, United States
Location: Grand Ballroom 7
Abstract:
Just as a successful story pulls readers into the lives of its characters, an effective evaluation can transport its audience into the lived reality of the individuals who serve as its focus. Yet neither of these works can come together without proper planning and development. In this session, attendees will explore how the steps for developing an effective evaluation plan can be artfully aligned with an author’s process for crafting a successful story.
When developing an evaluation plan for a program, project, or intervention, there are many elements that evaluators must consider. Similarly, authors have many practical considerations to attend to when developing a creative project. “Who comprises the cast of characters? What is the setting of the story? What do the characters experience that transforms them?” And, of course, “How will I get published?” and “Who’s going to read this anyway?”
Although the creative process of individual authors may differ, each storyteller aims to craft a narrative that engages their audience from beginning to end. Successful stories don’t simply emerge from the author’s mind in their final form! Rather, they are accomplished through an iterative process of planning and development. It is an author’s hope that, through this planning process, a well-designed story structure emerges that can guide their own creative process behind the scenes and propel their future audience through the story.
In much the same way, a thoughtfully designed evaluation plan can guide the work we seek to accomplish in the field of evaluation, not only serving as a roadmap for the evaluation work itself but also as a framework for telling the story of the evaluation project. Just as an author must consider their future readers, we as evaluators must intentionally seek the best methods for engaging our audience of stakeholders—whether they are community members, organizational leaders, project funders, legislators, or otherwise.
Join us for this interdisciplinary session as we explore how world-building and evaluation capacity-building align! This skill-building workshop will provide evaluation planning resources for attendees and invite them to apply the skill to their own relevant projects in real time. We hope you’ll leave with a new conceptual tool for your next evaluation… or perhaps an idea for your first novel! No writing or evaluation experience is necessary to attend. Come discover the storyteller in you!
This session will be led by Benjamin Williamson, who works as an internal evaluator for the K-12 STEM education non-profit CYBER.ORG. In his spare time, Benjamin enjoys many creative endeavors and is an award-winning screenwriter and filmmaker.
Relevance Statement:
The field of evaluation professionals includes a diverse range of individuals whose backgrounds may or may not include formal training in evaluation. By framing the drafting of an evaluation plan as analogous to the process used by authors to develop a story, I seek to make the field of evaluation accessible to a wider audience and less intimidating to evaluators who approach the field without formal evaluative training.
This session was inspired both by the conference theme, “The Power of Story,” and by my own work as a narrative storyteller, which I most often seek to accomplish through the medium of short film. In each of these areas, I know how important it is to have a plan to guide your process.
This session will prove valuable to the audience by connecting the common elements of a story to the considerations that we as evaluators must make. For example, an author may ask: “Who comprises the cast of characters? What is the setting of the story? What do the characters experience that transforms them? How will I get published? Who’s going to read this anyway?”
In the context of evaluation, these questions may take shape as: “Who (or what) is being evaluated? What is the context of the evaluation? What programs and/or initiatives are being evaluated and how do they attempt to transform the population they seek to impact? How will I report my findings? What is my audience of stakeholders?”
As this session is grounded in the process of developing an evaluation plan, I will refer to several resources that may prove helpful to those who are looking to develop or refine their own practices relating to this process. Resources will include: “How to Develop a Monitoring and Evaluation Plan” by Compass, The Community Tool Box by the Center for Community Health and Development at the University of Kansas, and the Evaluation Guidelines published by the United Nations Development Programme’s Independent Evaluation Office.
While each of the AEA Evaluator Competencies will be woven into the session to a degree, the primary competencies reflected here are Methodology, Context, and Planning and Management, specifically standards 2.1-2.5, 2.8, 3.1-3.2, 4.1-4.2, and 4.6.
References:
American Evaluation Association. (2018). (publication). AEA Evaluator Competencies Report. Retrieved March 21, 2023, from https://www.eval.org/About/Competencies-Standards.
Compass. (2022, December 12). How to develop a monitoring and evaluation plan. The Compass for SBC. Retrieved March 20, 2023, from https://thecompassforsbc.org/how-to-guide/how-develop-monitoring-and-evaluation-plan.
Hampton, C. (n.d.). Chapter 36, Section 5. Developing an Evaluation Plan. In The Community Tool Box. Retrieved March 21, 2023, from https://ctb.ku.edu/en/table-of-contents/evaluate/evaluation/evaluation-plan/main.
United Nations Development Programme. (2021). UNDP Evaluation Guidelines. United Nations Development Programme Independent Evaluation Office. Retrieved March 21, 2023, from http://web.undp.org/evaluation/guideline/index.shtml.