Evaluation Manager University of Iowa, United States
Abstract Information: The Great Plains Center for Agricultural Health (GPCAH) is one of 12 agricultural research, education, and prevention centers funded by the National Institute for Occupational Safety and Health (NIOSH) – a part of the Centers for Disease Control and Prevention (CDC). Similar to other federally-funded programs, current evaluation efforts are cross-sectionally focused – i.e., they assess the impact of programs in discrete funding cycles. However, this is likely to underestimate the true impact over time where multiple projects are undertaken with many partners within a particular portfolio and the impacts occur over many years as a result of the convergence of efforts. Drawing on two theory-based frameworks – Contribution Analysis and Translational Research Staging – this paper develops a longitudinal approach to evaluate the impact of two GPCAH priority program portfolios (1) agricultural equipment and road safety, and 2) air quality in livestock operations) over successive five-year funding cycles.
Relevance Statement: Workers in agriculture, fishing and forestry experience the highest fatal injury rate at 23.4 deaths per 100,000 full-time workers, compared to a rate of 3.5 deaths per 100,000 workers for all U.S. industries. The Great Plains Center for Agricultural Health (GPCAH) is one of 12 agricultural research, education, and prevention centers funded by the National Institute for Occupational Safety and Health (NIOSH) – a part of the Centers for Disease Control and Prevention (CDC). GPCAH’s goal is to reduce the burden of workplace injury and illness among farmers in nine Midwestern states by conducting research and surveillance, educating, trans¬lating research to practice, and communicating agricultural safety and health information and prevention strategies. Two current priorities of GPCAH include: 1) reducing traumatic injuries from roadway crashes with agricultural equipment, and 2) protecting livestock workers by reducing exposures to indoor air contaminants.
Similar to other federally funded research programs, GPCAH (and other NIOSH agricultural centers more broadly) face unique difficulties in showing the impact of their work. One reason for this is that center-level evaluation efforts are typically linked with discrete funding cycles and thus, measure only a part of the translation process of moving from research to practice (and eventually to mortality and morbidity outcomes), which is often very lengthy. In other words, a cross-sectional evaluation approach is likely to significantly underestimate the true impact of federally funded agricultural centers.
To address this concern, evaluators can draw upon theory-driven frameworks, such as Contribution Analysis (CA) and Translational Research Staging, to develop a longitudinal approach to evaluating the impact of NIOSH Ag Centers at both, portfolio (e.g., air quality or road safety) and aggregate-levels. When the unit of analysis is a program in a given funding cycle, the CA approach emphasizes the importance of assessing a program as an influencing factor in the theory of change. This is accomplished by assembling enough evidence to reduce uncertainty about the association, to reach “plausible association” – instead of a causal attribution – between the program and the desired end outcome(s). One way to approach this is by employing a “utilization-focused” lens where “plausible association” is demonstrated by adoption of research findings/products by relevant program stakeholders (e.g., employers, communities, trade associations). In this paper, we contented that at the portfolio-level, this approach could indicate the “adoption” of research findings by Principal Investigators of programs from the same portfolio in successive funding cycles. Concurrently, Translational Research Staging models provide scaffolding to this approach by laying out a multi-phase framework demonstrating the temporal movement from research to practice over time. In particular, the NIOSH translational research process comprises the following stages: Hazard/Problem identification; Development; Testing; Institutionalization; and Evaluation. In this paper, we apply key concepts from CA and NIOSH translational research process staging to longitudinally examine and demonstrate impact of GPCAH’s air quality and road safety portfolios over multiple five-year funding cycles.
