Collaborative, Participatory & Empowerment Evaluation
Shannon Gatewood, MPH, PMP
Director of Strategy, Policy, and Operations
W2 Consulting Corporation, United States
Chizoba Chukwura, MPH, CPH (she/her/hers)
Senior Consultant
W2 Consulting Corporation, United States
Chizoba Chukwura, MPH, CPH (she/her/hers)
Senior Consultant
W2 Consulting Corporation, United States
Arjee Restar, PhD, MPH
Director of Research and Evaluation
W2 Consulting Corporation, United States
Raul Almazar, RN, MA (he/him/his)
Senior Public Health Advisor
NASMHPD
Barrington, Illinois, United States
Joan Gillece, PhD
Director - Center for Innovation in Health Policy and Practice
NASMHPD, United States
Location: White River Ballroom E
Abstract Information:
Background: For many, trauma has shaped their daily lived experiences and worldview. Understanding the narratives of individuals participating in a trauma-informed program is crucial to program development and adaptation, and in evaluating program impact and outcomes. In evaluating TAMAR (Trauma, Addictions, Mental Health, And Recovery) – a NASMHPD-supported program that aims to broaden mental health support services delivered across several settings outside of behavioral health clinics (e.g., church, community centers, correctional facilities) by combining psychoeducational modules with self-regulation strategies for trauma symptoms – our research and evaluation team applied an integrated approach to program evaluation that is centered in storytelling to make a change in participants’ wellbeing.
Methods: Using participant and peer facilitator perspectives via in-depth qualitative interviews, pre- and post-test surveys, document reviews, and input from TAMAR leadership, we present lessons learned and challenges observed from our research and evaluation team’s integrated approach to storytelling. Specifically, we integrated lenses of intersectionality, human-centered design, and the stages of change to center and contextualize the stories of TAMAR participants and facilitators as they relate to TAMAR’s individual- and program-level outcomes and impact on (a) awareness and knowledge of trauma, (b) recognizing trauma symptoms, (c) use of trauma-informed coping skills for healing, and (d) building connections and relationships. Evaluation Challenges and Lessons Learned: To highlight the effects of TAMAR via storytelling, our team challenged the rigors of program evaluation methodology and learned to: (a) identify and prioritize impacts and outcomes that are deemed most meaningful to TAMAR participants, peer facilitators, and leadership as they relate to TAMAR program goals, (b) develop evaluation questions that measure stages of change and allow for individual reflexivity of trauma experiences, accounting for intersectionality of race, age, gender, class, and sexual orientation and how marginalization along these lines shapes their view and experiences of the world, during and after trauma, and (c) provide bi-directionality in evaluation material development and data sharing to foster a sense of ownership. Some challenges identified included: (a) time constraints based on established training schedules, (b) identifying opportunities for evaluation activities without disrupting the program and its priorities, and (c) developing novel evaluation measures that best capture the narratives of the participants and peer facilitators as they relate to program priorities. Altogether, these lessons learned and challenges encouraged our team to ground our work in storytelling, as well as value the voices and narratives of TAMAR participants, peer facilitators, and leadership over evaluator perspectives.
Discussion: Evaluating TAMAR showcases how leveraging storytelling and multiple, integrated strategies can be useful in delineating and highlighting program outcomes and impacts at the individual and programmatic level. This approach creates opportunities to apply lenses of intersectionality, human-centered design, and stages of change to qualitative and quantitative evaluation techniques and centers the evaluation on the lived experiences and reflections of program stakeholders. Understanding ways to best reconcile evaluation expertise with community needs can yield and increase rigor to evaluation activities while propelling the program towards building an evidence base.
Relevance Statement:
Stories are a powerful tool we can leverage for change, insofar as they’re told. In the first of what we hope will be a multi-phase effort, W2 Consulting and NASMHPD partnered to tell the story of a group that is not often the focus of program evaluation: training facilitators with lived experience. Our team utilized intersectionality and human-centered design approaches to focus on the narratives of facilitators who completed training on Trauma, Addictions, Mental Health, and Recovery (TAMAR) – a novel psychoeducational trauma-informed care (TIC) program. Through document review, observation, and facilitator surveys, we explored the goals and objectives of TAMAR, what the program means to the facilitators who received training, and how the program might have a broader impact on similar populations and settings. Join us as we share what we learned from the program developers, policymakers, advocates, and everyday people that care deeply about the psychological health and well-being of communities throughout Baltimore City.