Senior Researcher IMPACT, Inc. Milwaukee, Wisconsin, United States
Abstract Information: Growing a community-based coalition is a complex, challenging, and messy endeavor. It never occurs in a vacuum and rarely follows a predetermined plan (despite the best of intentions). Standard evaluation approaches can therefore be ill-suited for these types of collaborative initiatives. Community-level outcomes typically take years to be observed and measured; process evaluations often focus solely on tracking activities and outputs; and formative evaluations tend to reflect inward on a coalition’s functioning, membership, and implementation of strategies. How can an evaluation best tell the story of a coalition and its work in a way that reflects the community context, acknowledges the importance of emergent challenges and opportunities, and celebrates the many baby steps required to advance the work? This session will illustrate how ongoing documentation in an implementation log can help community-based initiatives tell richer stories about their birth, growth, and the significant moments (both big and small) along the way.
Relevance Statement: Over the last decade, systems-change approaches for addressing large scale community issues have gained popularity in the public sector, and the evaluation field has responded by rethinking frameworks and methodologies (Cabaj, 2018; Preskill & Gopal, 2014). These evaluation approaches seek to more fully capture the complexity of initiatives, demonstrate multiple levels of impact, and support ongoing learning and innovation. Storytelling and other qualitative methods fit naturally into systems-change evaluation frameworks, as they can shed light on contextual factors, points of influence, relationships, and other dimensions (Congdon, Simms, & De Vita 2020). As a data collection method, implementation logs can be an accessible way for project staff and evaluators to document events and activities on a day-to-day basis, rather than relying solely on the individuals involved in an initiative to recall and make value judgments about what was important upon later reflection (Adams et al., 2017; Shields-Zeeman et al., 2020). Adams, E. J., Cavill, N., & Sherar, L. B. (2017). Evaluation of the implementation of an intervention to improve the street environment and promote walking for transport in deprived neighbourhoods. BMC Public Health, 17(1), 655. Cabaj, M. 2018. Evaluating Systems Change
Results: An Inquiry Framework. Tamarack Institute. Congdon, W., Simms, M., & De Vita, C. (2020). Assessing the impact of community-level initiatives: A literature review, OPRE Report #2021-4, Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. Preskill, H., & Gopal, S. (2014). Evaluating complexity: Propositions for improving practice. FSG. Shields-Zeeman, L., Petrea, I., Smit, F. et al. (2020). Towards community-based and recovery-oriented care for severe mental disorders in Southern and Eastern Europe: Aims and design of a multi-country implementation and evaluation study (RECOVER-E). International Journal of Mental Health Systems. 14, 30.
