Abstract Information: Evaluators are often in control of the story that gets told through evaluation findings. We decide – in consultation with the client – what to include in reports, what to exclude, and how to structure analysis and reporting. But as evaluation consultants, we are increasingly hearing from (some of) our clients that they want more – more access to their data, more flexibility to ask different questions of their data, and greater ability to determine the types of stories that might be narrated and constructed for various audiences. For instance, client organizations may wish to draw on their own internal capacity to explore and query data sets to inform programmatic decision-making, reflect on progress, and share feedback with constituents. This presentation is a case study for the use of a data warehouse that stores cleaned, de-identified data from multiple sources collected as part of longitudinal work with a community school program. The program partners with schools, families, and the community to ensure that every student is prepared for high school, college, career, and life.
The presentation will feature a discussion of critical decisions made in the process of developing the data warehouse – and associated limitations – and the use of modern data management tools to do this. It will also engage with potentiality, imagining generative overlaps of data warehousing with visualization and the use of the two most common BI tools, PowerBI and Tableau.
Relevance Statement: Evaluators work hard to deliver maximum evaluation utility, but too often the resources spent on study design, data collection, analysis, and reporting do not usefully live on after the reporting stage. Data warehousing is an approach to data analysis and storage that maximizes the use value of collected data for users/stakeholders and clients by permitting secondary (including longitudinal) analysis outside of the regular evaluation lifecycle, and in response to new areas of interest and inquiry. It subtly shifts the balance of power between evaluators and those who commission evaluations by enabling commissioners and stakeholders to engage directly with cleaned and de-identified data to construct their own meaning and story. This maximizes benefits to those who provide data for the evaluation while minimizing risks, and represents an instance of the ethical use of technology in evaluation practice.
