41 - Letting the Data Speak: Effectively integrating and synthesizing your data to tell the story

Abstract Information: Over the years, we have developed an approach that efficiently and rigorously integrates and interprets data. We call it the Data Analysis, Integration, and Synthesis (DAIS) process. This participatory approach allows us to integrate complex data streams, often collected and analyzed by multiple evaluators. The DAIS process is interactive and iterative. During a DAIS process, we put analyzed data visually in front of the evaluation team (using cards or sticky notes), and we work together to integrate it into initial findings. We have found that this process can overcome the challenge of moving from analysis to report writing and can help lead to a well-structured report with strong findings, a clear narrative reflected in the conclusions, and recommendations grounded in data.

In this workshop, we present the DAIS process and invite the participants to practice each step. Participants will experience participatory strategies during this workshop to ensure data are effectively synthesized. We first present an introduction to the approach through a mini lecture and discussion around structuring a DAIS workshop. What is it? What are the benefits? Who needs to be there? How will it be formatted? We will introduce the process of creating data analysis summaries which comprise early emerging theme statements with supporting evidence and data sources. These are the key inputs to a DAIS. We will present a mock study with 4 key evaluation questions, and mock supporting data. Participants work in small groups to practice developing emerging theme statements for each question. We will then discuss how to prepare for a DAIS workshop. We will practice the workshop steps, focusing mostly on how an evaluation team can affinity map emerging themes into higher-level findings during a DAIS workshop. We will then discuss the difference between findings, conclusions, and recommendations and practice developing strong conclusions and recommendations from the emerging findings. Finally, we will discuss good practices and strategies to move from the DAIS to drafting a cohesive narrative in an evaluation report. We also share our techniques to conduct the DAIS process virtually. Throughout, participants will experience how DAIS processes produce traceable, contextualized and supportable evidence-based findings and user-appropriate conclusions and recommendations, with a confident team standing behind the story the data tell.

Relevance Statement: Often in evaluation, there is immense focus on data collection without fully considering the crucial analysis and synthesis part of the process. All evaluators have probably, at one time or other, found themselves with a mountain of evidence from various data streams, and struggle to figure out how to tell a strong story in an evaluation report. This can especially be an issue if multiple analysts and report writers are on the team, often in different locations. Some might also have concerns about bias if checks are not established along the way, or if the team moves from data collection to writing without fully synthesizing the data. The consequence can be a poorly written evaluation report that is not fully informed by the evidence. Over the years, we have developed an approach that efficiently and rigorously integrates and interprets data. We call it the Data Analysis, Integration, and Synthesis (DAIS) process. The DAIS process gives evaluation teams a structured and deeply participatory approach to pause after data collection and analysis, and work together in a workshop setting to integrate evidence and construct a meaningful story – including findings, conclusions, and recommendations- that are directly linked to the evidence. Participants will learn about the DAIS process and have an opportunity to practice each step of the process using mock data. The DAIS process brings several benefits. By pausing to have everyone on the evaluation team deeply engage with the data during a DAIS workshop, a more nuanced understanding of the evidence emerges, and findings are grounded directly in the evidence gathered. Each finding can be traced to specific evidence. The DAIS also ensures that the analysis and findings are grounded in the experiences of those involved in data collection. A DAIS can give the evaluation team a sense of ownership over the data and the analysis, leading to an increased confidence in the findings. Thus, the team feels more empowered to recommend action (if appropriate) based on the evidence. By involving the evaluation team in the DAIS process, the team can be confident that findings, conclusions, and recommendations are grounded in the evidence and presented appropriately.