Lessons Learned from Introducing Structural Change to a Change Adverse, Complex Environment

Abstract Information: This paper presentation will chronical the process and lessons learned from implementing phase one of the first ever collective impact evaluation system within Oklahoma State University’s (OSU) Cooperative Extension. Prior to the start of the project, OSU Extension had no formal long-term goals or meaningful evaluation capacity. Instituting either was complicated by many factors within the organization. All 77 Oklahoma counties served by OSU Extension house at least one Extension educator who develops and executes unique community-based programming. This creates a highly dynamic context that is responsive to community needs in the moment, but largely impossible to evaluate for long term, collective impact without the development of a new system. Implementing any kind of new system was further complicated by a fracted relationship between employees, different divisions, and leadership centered around a lack of trust, frustration at unclear expectations, frequent historical change to systems and processes, and a perceived disconnect between the lived realities of county-based employees and administrative demands. Any proposed change had to be handled delicately to not shock the system and cause widespread resentment and resistance. Data collection began in the summer of 2021 through several facilitated, iterative deep listening sessions with employees to better understand, from their perspective, what the ultimate goals of the organization are. These listening sessions also brought forth many stories about concerns and frustrations that were not directly tied to goal setting but important in the implementation phase nonetheless. Once a structure for goals and objectives was formulated from this feedback, the work turned to creating a system for organizing county programming that mapped onto existing organizational systems to clarify and streamline administrative responsibilities for overtaxed and overworked employees while also embedding and facilitating evaluative work. Finally, buy-in and unity in the approach needed to be reached by all divisions within Extension, something that had not occurred in recent institutional memory. It was only through careful listening and integrating of the stories, frustrations, concerns, and challenges of employees into the overall finished model that phase one was able to be implemented in December 2022 largely successfully. Work is currently underway to finetune phase one and respond to employee feedback in real time as well as to document lessons learned in the hopes of improving employee engagement in the future.

Relevance Statement: There is a lack of information available about starting a full fledge evaluation system from the ground up. What information is available largely comes from the those working within organizations that have more centralized structures than the county model of Extension education and organizations that are more receptive, or at least curious, about the value of evaluation to their work. The complex system my paper presentation describes was neither. It is a system rooted in 150 years of history and tradition that hasn’t needed evaluation and direct hostility towards any change, especially that handed down from leadership. Simply put, leadership brought an internal evaluator in, but the vast majority of the organization didn’t welcome, want, or see the value in that change.
As evaluation continues to grow as a field and gain prominence as a useful tool for organizational development, more evaluators are likely to find themselves in similar positions with a mandate to implement structural change within hostile or change adverse organizations. It is important that we as practitioners share our stories of success and lessons learned with each other in the moment but also in preparation for the needs of future evaluators in a changing landscape. This paper presentation attempts to step forward and admit change can be hard and unwelcomed, but with a methodological approach that listens and responses to tensions while also taking the role of negotiator and advocate for all parties involved, it can be done neutrally or beneficially.