45 - Using network science to tell the story of coalitions and collaboratives

Abstract Information: Background. Community coalitions and collaboratives are a common approach to addressing complex social and community problems. These groups mobilize cross-sector, inter-organizational members’ collective resources to understand problems, make decisions, and move towards action over time (Lawlor et al., 2022). Although these efforts are common, there is insufficient guidance about how to effectively do this work without creating additional burden on already-overburdened community organizations and systems (Varda & Sprong, 2020). Social network analysis can provide insights into the processes by which organizations partner with each other and how those partnerships relate to outcomes.

The workshop. This workshop will provide a comprehensive introduction to using network science to evaluate coalitions and collaboratives. No previous experience with social network analysis is necessary for participants to benefit from this workshop. The workshop will include four activities:

1. An overview of the basics components of network science and the network way of thinking,
2. A conceptual introduction to evaluating collaboratives with validated measures and methods using social network analysis to tell the story of their work,
3. A connection to practice with a focus on how these basic components of evaluating networks lead to actionable insights to develop network strategies,
4. A demonstration and training using PARTNER CPRM, a software platform designed for capturing, analyzing, and making sense of network data with community collaboratives.

The workshop will be split into two parts. Part one will cover an introduction to network science (activities 1-3) and hands-on training in conducting social network analysis using the PARTNER CPRM (activity 4). In part one, we will outline how network science contributes to both process and outcome evaluation with coalitions and collaboratives. We will then introduce the basic metrics of networks and show how evaluating these leads to insights. In part two, participants will engage in hands-on practice as they learn to use the PARTNER CPRM Platform. Participants will practice survey setup, data collection, analysis, and sensemaking, with a focus on translating data to practice. We will conclude with ways to use network data from tools like PARTNER to:

- Help coalitions and collaboratives to understand and tell the story of their partnerships using individual network member profiles as well as representations of the whole network,
- Link network data to outcomes
- Build network strategies that are actionable and effective.

Outcomes. Participants will leave with the knowledge to be informed consumers and users of social network analysis. They will be able to identify when it is appropriate to use network science for evaluation, the basics of setting up data collection and analysis, available tools to make evaluations streamlined and replicable, and approaches to reporting that support insights and action.

References

Lawlor, J. A., Metta, K. R., & Neal, Z. P. (2022). What is a coalition? A systematic review of coalitions in community psychology. Journal of Community & Applied Social Psychology, 32(1), 3–18. https://doi.org/10.1002/casp.2554

Varda, D. M., & Sprong, S. (2020). Evaluating Networks Using PARTNER: A Social Network Data Tracking and Learning Tool. New Directions for Evaluation, 2020(165), 67–89. https://doi.org/10.1002/ev.20397

Relevance Statement: Coalitions and collaboratives are a common approach for addressing a wide range of social problems. These efforts need to be evaluated in a way that captures their interacting parts and how they contribute to impact. Social network analysis is an approach that can capture the complexity of coalitions and collaboratives with data that leads to insights. This session will build participant skills for evaluating coalitions and collaboratives using network analysis. At the end of the session, participants will have the skills to (1) identify the need for relational data analysis in coalition and collaborative evaluations and (2) begin using social network analysis in their evaluation practice.

This session will attend to several of AEA’s guiding principles and program evaluation standards, including the systematic inquiry principle, utility standards, and accuracy standards. We will focus on the systematic inquiry principle by providing guidance about using social network analysis in a methodologically rigorous and contextually relevant manner. Specifically, we will train participants to consider key issues in data collection, analysis, and communication of findings to ensure a systematic process and useful results. We will also introduce participants to software that can facilitate data collection, cleaning, and analysis to ensure a systematic, replicable process.

We will engage in the utility standards for evaluation (standards U2, U3, U6, and U7) by providing skill-building to determine when it is appropriate and useful to implement social network analysis in a given evaluation context and the types of stakeholder needs it can meet. This session will also include a strong focus on the accuracy standards for evaluation. We will teach participants about developing a sound design to support sound conclusions (standard A6), to capture valid and reliable data using validated survey questions (standards A2 and A3), to manage their data using a platform built to track social networks for evaluation purposes (standard A5), and to effectively communicate and report findings to coalitions, collaboratives, and other stakeholders (standard A8).

In sum this session creates value by building participant capacity to to think like a network scientist and to apply social network analysis in evaluation practice while attending to stakeholder needs and ensuring methodological rigor. The training will be driven by a focus on systematic inquiry, utility standards, and accuracy standards.