12 - Integrating partner voices in multi-site evaluation planning

Abstract Information: Engaging partners is an important step in evaluation planning, particularly when using an equitable evaluation approach. Gathering multiple perspectives can help tell a more complete story of the program and the people it serves. However, it is challenging to integrate multiple voices into evaluation planning for multi-site and multi-level programs. We partnered with a statewide commercial tobacco control program in Indiana to develop an evaluation plan for a program that spans 60+ organizations. In the first phase of the project, we created a plan from conversations with program leadership and an extensive document review. During that time, we identified the need to gather input from individuals who frequently contribute to and use evaluation findings, with a focus on partners that serve communities disproportionately impacted by tobacco and targeted by tobacco companies. This presentation focuses on the second phase of the evaluation plan. We worked with program leadership to identify key partners that represented a range of sectors and communities and conducted virtual interviews to inform statewide evaluation planning. Interviews asked about partners’ current data uses and needs, challenges, future data needs, and strategies to engage communities with evaluation. Program leadership used partner feedback to improve data collection and dissemination strategies. In this session, we will share our successes and challenges gathering partner feedback to inform statewide evaluation planning and facilitate a group discussion during which attendees can share their own experiences. Discussion questions will focus on strategies to engage partners, challenges, solutions, and resources.

Relevance Statement: It is important to engage partners in all stages of evaluation, especially when using an equitable and anti-racist approach to evaluation.1-2 Including a range of perspectives and voices can help ensure that evaluations are designed and implemented equitably. However, multi-site and multi-level evaluations often encompass a large number of decision-makers, staff, participants, and partners that can span geography, sectors, and organization types. It may not be feasible to include every one of them in the evaluation design, particularly with a limited scope and evaluation budget. With this presentation, we seek to share our experience gathering input from multiple partners to incorporate into evaluation planning, with a focus on how data is collected and reported. We worked with the Indiana Tobacco Prevention and Cessation (TPC) program to conduct interviews with organizations across the state and at multiple levels. Interviewees included those currently receiving funding from TPC to conduct commercial tobacco control activities, as well as organizations that partner with TPC but do not currently receive funding. We selectively sampled individuals that represented a range of communities and sectors. By sharing our experience, we hope to provide a real-world example to other evaluators of gathering and using partner feedback for multi-site evaluation planning, including our successes and challenges. Specifically, we will describe how we used an equity approach to identify and interview partners within a program that spans over 60 organizations across the state. We also hope to hear from session attendees about their own experiences, challenges, solutions, and resources, and facilitate a group discussion around this topic. Our goal is for attendees to learn practical steps to integrate partner voices into evaluation planning using an equity approach, discuss challenges we have faced, and collectively brainstorm solutions to address these challenges. This session will cover the AEA guiding principles of Systematic Inquiry and Respect for People by explaining our approach to designing, sampling, and interviewing partners to inform statewide evaluation planning. It will address the AEA competencies of Context and Methodology. 1. WestEd. (2021). Anti-racist evaluation strategies: A guide for evaluation teams. https://www.wested.org/resources/anti-racist-evaluation-strategies/ 2. W. K. Kellogg Foundation. (2021). Doing evaluation in service of racial equity: Deepen community engagement. https://everychildthrives.com/doing-evaluation-in-service-of-racial-equity/