185 - Understanding Policy Impact from the Translational Science Context

Abstract Information: With the NCATs focus to advance the science of translation, which is the process of turning observations into interventions to improve health, informing public policy is one critical way to improve the health of individuals and communities. To understand the context of translation at BU, evaluators are working with individual investigators, school deans, and faculty development committees to understand the goals, needs and culture of the academia setting at BU. Along with understanding, what counts as “translation” in each discipline across BU. Narrative data has always been integral to evaluation (e.g., ethnography, qualitative methods to do case study). It serves to illuminate context, build connection between different stakeholders, develop program evaluation theory, foster meaning making, and more. Evaluators are currently compiling interview data and developing a survey to send to all investigators within the BU community to understand “translation” and “policy impact success”. This interview and survey data will help to describe policy impact use, benefits, and opportunities for culture change within the CTSI context. BU evaluators are responding to the unique context of the policy landscape. We are engaging a diverse range of stakeholders from individual investigators in various disciples, to deans of multiple BU schools, to faculty development staff, while facilitating and communicating the evaluation process and results in an equitable manner. Overall, we are promoting policy impact evaluation use and influence within the CTSI context.

Relevance Statement: “Clearly an understanding of context is important to frame research impact assessments but generating data to inform such understanding may not be simple. The links between context and timing mean that context changes over time” (Morton, 2012). Narrative data has always been integral to evaluation (e.g., ethnography, qualitative methods to do case study). This project serves to illuminate context, build connection between different stakeholders, develop program evaluation theory, foster meaning making, and more. This work and methods is relevant to the current context of the evaluation within the translational science context. Evaluators are currently compiling interview data and developing a survey to send to all investigators within the BU community to understand “translation” and “policy impact success”. This data that will be compiled over the next months will help to describe policy impact, benefits, and opportunities for culture change within the CTSI context, which will be relevant and valuable to all evaluators in this context, along to evaluators outside of translational science. This proposal ties directly to AEA’s competencies, 3.0 Domain Context, which focuses on understanding the unique circumstances, multiple perspectives, and changing settings of evaluations and their users/stakeholders. BU evaluators are responding to the unique context of the policy landscape. We are engaging a diverse range of stakeholders from individual investigators from various disciples to deans of multiple BU schools to faculty development staff, while also facilitating and communicating the evaluation process and results in an equitable manner. Overall, we are promoting policy impact evaluation use and influence within the CTSI context. To create discussion in this roundtable, we have included questions that would be posed to the audience to help better understand the Translational Science context and evaluation practices in this context: • How is policy impact measured in your context? • What tools are you utilizing to measure policy impact? • What challenges does the translational science context pose to measuring policy impact? Citations: • Mirvis DM. From research to public policy: an essential extension of the translation research agenda. Clin Transl Sci. 2009 Oct;2(5):379-81. doi: 10.1111/j.1752-8062.2009.00144.x. PMID: 20443923; PMCID: PMC5350749. • Morton, S. (2012) Exploring and Assessing Research Impact’, in Social Policy. Edinburgh: University of Edinburgh. • Annette Boaz, Siobhan Fitzpatrick, Ben Shaw, Assessing the impact of research on policy: A literature review, Science and Public Policy, Volume 36, Issue 4, May 2009, Pages 255–270, https://doi.org/10.3152/030234209X436545