Evaluation Managers and Supervisors
Hannah Kitchin, MA
Evaluator
L&G Research and Evaluation Consulting, Inc.
New York, New York, United States
Hannah Kitchin, MA
Evaluator
L&G Research and Evaluation Consulting, Inc.
New York, New York, United States
Danielle Campbell, DrPH, MPH (she/her/hers)
President/Co-Founder
L&G Research and Evaluation Consulting, Inc.
New York, New York, United States
Danielle Campbell, DrPH, MPH (she/her/hers)
President/Co-Founder
L&G Research and Evaluation Consulting, Inc.
New York, New York, United States
Location: Room 103
Abstract Information: Across all fields, there is an inherent desire to tell a data driven story that demonstrates the short-term and long-term impact of a program, intervention, or initiative. For example, providers of state or federally funded out of school-time programs are often expected to collect and report accurate participation data because of the strong evidence of the association between program attendance and gains in outcomes related to academic achievement, school day attendance, and social and emotional learning. However, measuring the success and improving the management of these programs, interventions, or initiatives is predicated on strong monitoring and evaluation (M&E) systems that produce quality data by utilizing data entry and management systems correctly and maintaining accurate program records. This process can be particularly challenging in the early stages of implementation because of staff capacity, funder requirements and expectations, and competing priorities. One way to support the establishment of effective M&E systems is the routine utilization of data audits. This process facilitates the assessment of data for quality throughout its lifecycle to ensure its accuracy and efficacy to support continuous program improvement efforts and the assessment of impact. To help programs build their data literacy to assess the quality of their data and to strengthen their data management and reporting systems, L&G Research and Evaluation Association Inc. developed a Data Audit Tool that measures the accuracy of out of school time program attendance records. The objective of this tool was two-fold: 1) to verify the quality of reported data for key performance indicators; and 2) to assess the ability of data management systems to collect and report quality and accurate data. This demonstration will provide an overview of the importance and benefits of establishing a data audit protocol; how to incorporate a data audit tool into a program’s larger M&E plan; how and when to engage stakeholders in the process; and the key design principles to consider when designing a similar evaluation tool for other types of programming. The use of an effective data audit tool and protocol can be a powerful instrument that helps programs confidently tell their story with both accuracy and integrity.
Relevance Statement: Data accuracy is a key component when conducting participatory evaluations measuring program impact. In grant funded youth development programs, programming is often implemented by community-based organizations (CBOs) that are well-positioned to serve their community but may not realize the importance and connection between effective data collection and management methodology and program data to accurately tell a program’s story. This disconnect makes it necessary for the evaluator to bridge the gap and build program capacity around data entry in ways that align with best practices as well as funder requirements. Data audits are an effective way to facilitate this learning process with program partners because it meets partners where they are while providing clear guidance on how to improve. This demonstration will walk attendees through the development and use of a data audit tool designed to assess the accuracy of out of school time program data. It will also detail how to leverage data audits to build data literacy skills at the program level and to help increase a program's capacity to incorporate data into self-assessment and improvement processes related to a comprehensive monitoring and evaluation plan.