Disaster & Emergency Management Evaluation
Justin Lawrence, MPA
Senior MEL Associate
Resonance Global, United States
Justin Lawrence, MPA
Senior MEL Associate
Resonance Global, United States
Olga Schetinina, MA
Senior Evaluator
Resonance Global, United States
Kaitlan Havlicek-McClenahan, MA
Senior Analyst
Resonance Global, United States
Zumrat Salmorbekova, MA
Team Leader
Resonance Global, United States
Sarah McLaughlin, MA
Senior MEL Specialist
Department of State, United States
George Mesthos (he/him/his)
Ukraine Program Officer
Department of State Bureau of Conflict and Stabilization Operations
Beverly, New Jersey, United States
Location: Grand Ballroom 7
Abstract Information: An evaluator holds the unique position of storyteller as well as arbiter in storytelling. This panel found itself in this very position when looking at programs being implemented in conflict affected Ukraine. First, panelists will discuss findings from a veterans’ reintegration program, which, started before the full- scale invasion, and then our team will discuss how we safely collected stories about the needs of veterans who often went back to the frontlines to protect Ukrainian independence. Second, we will discuss our most recent findings from the Cohesion Youth Led Action Program (CYLA). Another program prior to the invasion, this program focuses on internally displaced persons (IDP) youth leadership, mentorship, and integration of IDP needs and interests into local development programs to strengthen social cohesion at the community level. As a response to the invasion, many young IDP women and men participated in storytelling workshops, which helped them both articulate their feelings under direct circumstances and share visions of hope and prosperity in a post war Ukraine. The panel will discuss how we detected outcomes based on these stories. Finally, our clients at the DoS Conflict and Stabilization Office (CSO) will discuss how they take complex evaluations and distill them into “stories” about programs as a way to communicate and share programming successes across the entire Department.
Relevance Statement: Our topic is particularly relevant because it demonstrates the mutually reenforcing way that storytelling and evaluation influence one another. We plan to present two evaluation cases that collected stories about targeted populations to learn their experiences and needs, and use these stories to help a client to make informed decisions and share the narrative with important decision-making audience. The session will contribute to the field of evaluation practice by illustrating how to both use storytelling as an evaluation data collection device (Vets), how we can evaluate storytelling done through newer mediums such as podcasts and digital content (CYLA), and how storytelling can be used as a powerful and effective evaluation dissemination tool (both). The first case will discuss a midterm performance evaluation of the Department of State’s Conflict and Stabilization Office Veterans Reintegration Program. In this work and upholding Do No Harm, the team spoke with veterans during the full-scale invasion to collect their own stories and experiences as a way to frame and inform the needs of a relatively new population in Ukraine, recent combat veterans. At the onset of the program there was an estimated 400,000 Veterans largely from fighting in 2014, and that number is now estimated to land between 1.5-2 million people. We will share how we used storytelling to give faces behind these staggering numbers of need. The second case study will discuss how we evaluate storytelling as a tool of empowerment, advocacy, and civic activism, and the effect we observed in Ukrainian IDP and host community youth led initiatives at the community level. We are currently conducting a final evaluation of the Cohesion Through Youth-Led Action Program (CYLA), in which Ukrainian IDP and host community youth are given tools to become active members in their community, promote tolerance and advocate for integration of IDP youth needs into the local development strategies to strengthen social cohesion. In many instances, the invasion has forced IDP youth to relocate yet again, but the program pivoted towards using storytelling as a way for youth to both process their experiences, develop their impact stories, and articulate their vision for a post war Ukraine. Lastly, we discuss how evaluation results, which can be complex and rigorous, are distilled in succinct and accessible stories to help inform decision-making. In this last topic, we discuss how the commissioner of evaluations discussed how they used insights form both of the above evaluations to be share the story of their programming to important audiences like Congress and Office Leadership.
Presenter: Justin Lawrence, MPA – Resonance Global
Presenter: Kaitlan Havlicek-McClenahan, MA – Resonance Global
Presenter: Zumrat Salmorbekova, MA – Resonance Global
Presenter: Olga Schetinina, MA – Resonance Global
Presenter: Sarah McLaughlin, MA – Department of State