60 - Principles in Practice: Developing mixed methods measurements for principles-focused evaluation in agroecosystems research

Abstract Information: ‘Principles-focused evaluation’ provides a set of strategies to develop guiding, values-based statements to use in the evaluation of complex, adaptive, mission-driven programs (Quinn Patton 2018). However, measuring the adherence to, application of, and impact of those principles remains a challenge for large transdisciplinary programs, particularly those operating with a variety of timelines, funders, and priorities. This poster will discuss participatory approaches to both identify and measure evaluative principles, highlighting examples from several USDA-funded agroecosystems research groups. We will share the use of principles as project management and communication tools, and go over mixed methods to systematically measure how principles impacted the effectiveness and emerging outcomes of these projects. Ranging from developing perennial crops to increasing livestock grazing to integrating solar panels in farming, these agroecosystems projects bring a wide range of disciplines and stakeholder perspectives to the table. We will share applications of several different data collection methods with a principles-focused evaluation lens, including: Most Significant Change stories, Ripple Effects Mapping, Evaluative Rubrics, Start-Stop-Continue workshops, and Key Informant Interviews. 

Relevance Statement: Principles-focused evaluation provides a powerful approach to identify meaningful stories of change within large, complex, adaptive programs and projects. Transforming agroecosystems in North America in the face of climate change and large-scale agricultural consolidation will require tremendous effort in the coming decades, and developmental strategies such as principles-focused evaluation will be critical in that change. Our team has used principles-focused evaluation to find common language and guidance between incredibly diverse disciplines, so that both plant breeders, farm managers, and agricultural policy-makers can reach shared understanding, direction, and sense of community within large transdisciplinary projects. Using largely qualitative, mixed methods approaches for data collection and analysis in principles-focused evaluation is critical to involve stakeholders and create shared ownership over values-driven agricultural research and extension. This session is particularly relevant to this years’ conference theme because of the focus on shared narrative and values in the mixed methods approaches discussed from our agroecological case studies. It will provide participants with tangible strategies and examples about how principles can shape project activities, evaluation data collection, and involve program stakeholders in identifying the most significant outcomes, effects, and stories of their projects.