No One-Use Visualizations: Making the Most of a Dashboard

Abstract Information: Data visualizations, especially those aggregated into dashboards, are increasingly relevant as powerful evaluation analysis, reporting, and dissemination tools. Strong dashboard development is time-consuming and labor intensive, so no visualization should be “one-use”. In this session, the presenters will discuss a process for building dashboards that: 1) serve as internal tools to support evaluation teams in analysis, 2) support client reporting requirements, and 3) can be used by clients for knowledge sharing and continuous improvement. As a working example, the presenters will focus on their evaluation of the Colorado READ Act, a law passed by the Colorado state legislature in 2012 aimed at improving student K-3 reading skills. The presenters will provide some context about this evaluation and will describe their experience developing dashboards for the evaluation. Central to this presentation is how the evaluation dashboard reflects the story of the READ Act, incorporating a wide array of data reflecting stakeholder experiences and outcomes in a way that lends itself to action and understanding. Presenters will discuss how, by building relationships with clients, relying on extant data, developing an understanding of the history of the data and its collection, utilizing iterative and flexible design schemes and processes, and crafting intentional, leveled data structures to power dashboards, other evaluators can also build multi-use visualizations that are relevant to a number of audiences and stay relevant throughout the life of an evaluation.

Relevance Statement: Data visualization tools, such as Tableau, are increasingly being used for evaluations as a data analysis tool, and as an accessible method of presenting findings to clients and broader audiences. This analysis and reporting tool is particularly beneficial, if designed effectively, as it can often serve multiple purposes for the evaluation (i.e., meet the needs of the evaluation team, the client, and broader audiences/stakeholders). Given the present-day focus for State Education Agencies (SEAs) to use dashboards both 1) internally to increase their knowledge of the actions and results of schools in their state, and 2) externally to provide stakeholders and/or the public with information that can be used to answer questions of interest, the presenters aim to use their experience with large-scale evaluations, and creating dashboards for these evaluations, to provide other researchers with the lessons the presenters have learned through the dashboard development process and the important steps to keep in mind while developing visualizations for an evaluation. During the presentation, we aim to provide attendees with key takeaways that they can use in working with clients to develop visualizations for an evaluation. We will provide our insight into developing effective relationships with clients that will facilitate the creation of useful visualizations, and we will emphasize the importance of understanding the “story” of an evaluation (and the data collected pertaining to the evaluation) in attempting to build an accurate “story” throughout and following the evaluation.