Using Participatory evaluation as a framework for evaluation planning and implementation to improve data communication

Abstract Information: This session will explore the potential benefits of implementing an alternative approach on an ongoing program evaluation. I-CASH is an organization whose mission is to improve the health and safety of Iowa's agricultural community. The current evaluation employs a hybrid methodology that combines outcome and process evaluation techniques. It focuses on determining I-CASH's impact and reach through a variety of evaluation activities. Evaluators develop and implement evaluation plans with minimal participation from program stakeholders. This presentation advocates the use of practical participatory evaluation (P-PE) to convey the I-CASH community's story more effectively in the evaluation findings. In P-PE, essential program stakeholders are more actively involved in the design and implementation of the evaluation. The presenter will describe the theoretical and practical basis for selecting P-PE for this project, the potential challenges and barriers, and how to use the P-PE framework to plan and modify the evaluation approach for future projects.

Relevance Statement: I-CASH, founded in 1990 by the Iowa State Legislature, is one of the only mandated, state-funded agricultural safety and health institutions in the country. I-CASH aims to: 1) improve safety culture on Iowa farms; 2) provide statewide leadership in agricultural safety and health; and 3) increase I-CASH's visibility in Iowa. The objective of the program evaluation is to determine the statewide impact and reach of current and historical I-CASH activities and resources; and identify opportunities for future enhancement.

The current I-CASH evaluation employs a hybrid approach that combines outcome and process evaluation methodologies. Evaluation activities include interviews, surveys and focus groups with a diverse group of stakeholders. The current approach is a traditional evaluator-centered method in which professional evaluators device and carry out the evaluation with minimal input from the key program stakeholders. Which may potentially reduce the usefulness of evaluation results. In this presentation, a different methodology is proposed for evaluating I-CASH. Considering the purpose of I-CASH's evaluation and the current comprehension of the program, the following three characteristics of I-CASH were identified: 1) as a state-funded program, I-CASH must perpetually demonstrate its value; therefore, the results of this evaluation are essential for I-CASH to enhance the program; 2) the key program stakeholders (program director and key personnel) demonstrate a propensity to support and actively participate in evaluation activities, which partially satisfies the evaluation's responsiveness requirement; 3) after a year-long evaluation, there is an ongoing need for internal evaluation of the program.

Participatory evaluation (PE) is suggested as an alternative evaluation framework for the current program based on the aforementioned program characteristics and to better tell the stories of the data collected. PE refers to applied social research involving collaborative relationships between professional evaluators and practice-based decision makers, organizational members with project responsibility, or individuals with a significant interest in the program (Cousins & Earl, 1992). This study specifically recommends the use of practical participatory evaluation (P-PE, P-PE is one form of PE that is practical and supports program or organizational decision making and problem solving, and it places the greatest emphasis on utilizing evaluation results (Garaway, 1995) for the following reasons: 1) P-PE could enhance the use of evaluation results through in-depth collaboration between the evaluator and key stakeholders; 2) P-PE provides the opportunity for primary stakeholders of the program to share ownership of the evaluation with professional or external evaluators; 3) the key stakeholders involved in P-PE would gain the technical capability to proceed with evaluation activities or evaluative ways of thinking afterwards.

The overarching picture of this proposal is to use the practical participatory approach to evaluate I-CASH, focusing primarily on the program impact and reach. Therefore, the main technics and concepts of a process and outcome evaluation would still be applicable, only in conjunction with P-PE. The presenter will describe the theoretical and practical basis for selecting P-PE for this project, the potential challenges, and barriers, and how to use the P-PE framework to plan evaluation activities for the second half of the project.