Learning the Stories Behind the Data: Engaging Stakeholders in Data-Driven Technical Assistance to Power Evaluation

Abstract Information: Capturing stories that provide context to data broadens the scope of evaluation. Engaging stakeholders who bring the data to life through stories builds investment and collaboration. The process becomes even more powerful when evaluators work directly with technical assistance providers helping stakeholders understand and apply data and pinpointing stakeholder strengths and needs. The technical assistance and evaluation teams at the National Coordinating Center for postsecondary education for individuals with intellectual disability in the United States recognized the need for a data driven technical assistance (DDTA) model to support and learn from program-level stakeholders at colleges and universities serving students through model demonstration programs. In addition to learning about recent program developments, addressing needs, and discussing annually reviewed data and outcomes, the DDTA team prioritizes annual initiatives across all programs based on specific programmatic features or student outcomes. The presenters will explain the objectives and development of DDTA, offer steps and considerations for implementing a collaborative approach to evaluation and technical assistance, and emphasize measures taken to prioritize the perspective of program-level stakeholders, capturing the stories and context behind their data, success, and needs. The presenters will then outline the follow-up procedures used to support stakeholders, meet technical assistance needs, and maintain engagement. The presenters will conclude by engaging attendees in discussion of implications of the DDTA model on evaluation practice and the applicability of the model in any evaluation context where the power of understanding data through stories, improving practice, and supporting and engaging stakeholders is valued.

Relevance Statement: The power of evaluation is evident in the resulting stories and data that drive decision-making, planning, and funding allocations. Ideally the impact of evaluation-driven decisions is evidenced in positive trends in future evaluation. Ma (2018) refers to the relational process in which data are interpreted as information adding to the knowledge base and expertise available on a topic leading to action by way of informed decision-making. The power of evaluation is further strengthened when evaluators take the important step to help stakeholders understand the benefits (Mason, 2022). Ma (2018) specifically advises on the leap from knowledge to action, emphasizing the importance of team collaboration around essential issues and major goals. Rising to the challenge of collaboration can lead to great success: stronger buy-in, robust participation in reporting, rich stories and qualitative input to drive evaluation, and the stakeholder benefit of more information and knowledge to drive action and initiatives. Involving stakeholders and evaluating collaboratively can help everyone continue the story together. The evaluation team at the federally-funded National Coordinating Center for postsecondary education for individuals with intellectual disability in the United States works closely with a team of content and practice experts providing technical assistance to the center’s stakeholders at colleges and universities across the nation. From these efforts the teams recognized the need to bring together the power of stakeholder participation, user-friendly technical assistance, and accessible evaluation to enhance program development and inform the field. The coordinating center developed and implemented a data-driven technical assistance (DDTA) model bringing together program personnel, technical assistance providers, and evaluators in productive collaboration. Recently completing a third annual cycle, use of the DDTA model has resulted in better stakeholder engagement, improved understanding and addressing of technical assistance needs, and enhanced story-driven evaluation efforts.While discussing annually reviewed data and outcomes, the DDTA team prioritizes annual initiatives across all programs based on specific programmatic features or student outcomes. The presenters will describe the recognized need for and development of the DDTA model and offer steps and considerations for implementing a collaborative approach to evaluation and technical assistance. They will emphasize efforts to prioritize the perspectives of program-level stakeholders and capture the stories and context behind their data, success, and needs. The presenters will then outline the follow-up procedures used to support stakeholders, meet technical assistance needs, and maintain engagement throughout the year. Finally, the presenters will conclude by engaging attendees in discussion of the implications of the DDTA model on evaluation practice and the applicability of the model in any evaluation context where the power of understanding stories, improving practice, and supporting and engaging stakeholders is valued. Ma, X. (2018). Data-information-knowledge-action model. In L. A. Schintler & C. L. McNeely (Eds.), Encyclopedia of big data (pp. 1–4). Springer International Publishing. https://doi.org/10.1007/978-3-319-32001-4_64-1 Mason, S. (2022). Just give me an example! Exploring strategies for building public understanding of evaluation. American Journal of Evaluation, 1–19. https://doi.org/10.1177/10982140211061018