Mixed Methods Formative Evaluation of Domestic Violence Guided by the Consolidated Framework for Implementation Research: Findings from the Community

Abstract Information: The Sexual and Domestic Violence in Survivors Project (SDVSP) aims to address domestic violence (DV) in child and adult survivors in a small under-resourced, multi-generational town in Massachusetts through inter-agency collaboration, community-based tailored intervention, and dedicated resources for sustainability. Engaging the community and agency stakeholders in uncovering gaps in services and barriers to utilization of existing resources is essential for creating new and expanding current programs in a community with ingrained stigma and misperceptions around DV. The Winthrop Department of Public Health & Clinical Services (WDPH&CS) partnered with Boston University School of Public Health (BUSPH) to conduct an unbiased evaluation to inform the development of DV-related resources and services. The mixed-methods formative evaluation by BUSPH includes (1) a stakeholder analysis to identify entities already engaged, and not yet engaged, in issues of domestic violence; (2) an analysis of stakeholders’ stage of change, factors for motivation, and power in the issue; and, (3) a partnership with WDPH&CS to address it. A stakeholder survey was sent to all identified stakeholders, including community-based organizations, schools, local health agencies, public safety, places of worship, and mental health clinicians, to uncover insight into the current landscape, perspective, and gaps related to domestic violence. A targeted qualitative approach (focus groups/interviews) guided by survey findings and grounded in the Consolidated Framework for Implementation Research (CFIR) examined internal context, individual characteristics, intervention design, external influences, and processes for addressing domestic violence by stakeholders including stakeholder agencies and community residents. CFIR was used not only in the data collection and analysis but in the reporting of recommendations and next steps for WPDH&CS. This allowed for actionable recommendations to enhance existing programming and inform new efforts. This process also brought collaborators together in a coordinated system to address DV within the town. The mixed-methods approach and evaluation findings are timely given the pandemic-related challenges faced by communities and the need for a multi-sector collaborative response. In addition, the use of CFIR for evaluation allows for an understanding of the local context and facilitators and barriers to implementation. Recommendations will enhance future iterations locally and can inform best practices for similar efforts.

Relevance Statement: This proposal addresses each of the conference’s two themes. First, this proposal highlights the value of engaging a variety of stakeholders using a mixed-methods formative evaluation to uncover stories. Through surveys that then inform the story-telling aspect inherent in focus groups and interviews, various stakeholders such as public safety, DV organizations, and community members were included in uncovering the needs within the community, the stigma that exists in accessing services, and the appropriate ways to message around DV and the associated services. The stories that emerged as a result of the formative evaluation offer context to the community and transcend data points. The mixed-methods nature of the formative evaluation enhances its ability to combine and connect stories from diverse perspectives with the survey data allowing greater representation of DV stakeholders and broader perspectives, while the qualitative data allowed for greater depth and context. Guided by an implementation framework (CFIR), the revealing stories from data collection provide multi-faceted recommendations to the community’s department of public health. Ultimately, the results provide an opportunity for the dissemination of invaluable lessons learned and recommendations. This is essential when considering interventions that have the potential to support the local public health of the community, especially when the interventions should be developed based on community-engaged efforts. Second, this proposal highlights the importance of stories in fueling evaluation efforts in the community. The mixed-methods nature of the needs assessment provides a variety of inputs from varied stakeholders. Formative evaluation data were directly provided by identified stakeholders, who offer invaluable knowledge and context to the needs of local communities. Formative evaluation participants ranged from public safety officers to community members to youth groups, all offering perspectives of the context within the community. To uncover stories from all key groups, the formative evaluation process seeks to dig deeper and provide the context of the community. Guided by the CFIR implementation framework, the collected stories and data support shaping the evaluation process as needed to include more context. Additionally, by utilizing CFIR, which allows for a thorough consideration of the outer setting (context), inner setting (agency), and plans for sustainability, the priority of the needs assessment is to begin with context to propose evaluation processes. This proposal highlights the importance of a mixed-methods formative evaluation guided by the CFIR framework to systematically assess multiple levels and scenes of DV within the community. The mixed-methods formative evaluation fueled principal stories and outlooks, essential in disseminating a narrative of the collected data. Perspectives of the identified stakeholders fueled the formative evaluation by providing the context and needs of the community.