Systems in Evaluation
Kathleen Sullivan, JD (she/her/hers)
Principal
Fine Gauge Strategy LLC
Lincoln, Massachusetts, United States
Thomas Kelly, MPH
Principal
KEL Advising
Palm Springs, California, United States
Rhonda Schlangen, MA (she/her/hers)
Learning and Evaluation Consultant
Rhonda Schlangen Consulting, LLC
West Pawlet, Vermont, United States
Location: White River Ballroom H
Abstract Information: In our increasingly volatile present, evaluators and evaluations need to consider multiple possible future states to broaden our understanding of contexts and assumptions. How do evaluators open up our aperture and perspective about what is possible (and not just probable) around the assumptions underlying our theories of change? Strategic foresight provides a rigorous and proven set of tools to perceive, make sense of, and act upon ideas about the future. The “alternative scenario method” uses possible futures that could arise as a result of the trends, events, wildcards, and models we are currently using, as well as our creative imaginations. This method–stories of the future–can be used at different stages in the lifecycle of an evaluation to stress test assumptions and findings. Through this experiential introduction, participants will learn to work with future scenarios to test hypotheses and assumptions at the early stage of constructing theories of change.
Relevance Statement: The past few years have demonstrated that our economy, climate, politics, and social order are capable of changing much faster than in past decades. Our present, and certainly our future, will continue to be volatile, ambiguous, complex, and uncertain (VUCA). For our evaluation clients, it is no longer enough to reflect on the past and the present in decision-making and program development. Evaluators have to actively anticipate what may happen in the future and feed that information back into decision-making and evaluation planning. In the past, during slower moving times, experience was a reliable guide and the future was an extension of the past. Now that the future is happening more quickly and involves tremendous change and uncertainty, organizations need a more systematic approach. “Strategic foresight” provides a rigorous set of tools that are readily adaptable for purposes of stress-testing and future-proofing the assumptions and hypotheses built into program models and theories of change used by US nonprofits, philanthropies, and governments. The power of strategic foresight is that it requires decision-makers to identify and grapple with multiple potential future states; not to predict the future but to build a culture of flexible thinking in which elements of VUCA are confronted as manageable challenges and not roadblocks. Strategic foresight does not replace but can complement existing strategic planning tools. It can help projects be more resilient by broadening the amount of information that is tracked and increasing awareness of emerging issues or signals of the future. Evaluators trained in foresight methods are well positioned to help clients be more forward thinking in their strategic planning. Strategic foresight enlists multiple approaches that are already familiar to many evaluators. Forecasts are statements about the future that are logical (versus predictions which are statements about the future that are intended to be accurate). Computer modeling creates different scenarios using multiple inputs and formulas. The Delphi survey gathers information on peoples’ thoughts about how the future is likely to unfold. And visioning can determine a long-term image of the future that organizations, communities and individuals can aspire to. The “alternative scenario method” is a basic foresight methodology that is both powerful and straightforward to deploy. Seventy years ago Herbert Kahn and the RAND Corporation developed the use of scenarios to assist in military and government strategic planning that could be informed by multiple possible futures. Alternative scenarios are stories about the future which force people to think creatively as well as envision a desired or aspirational future and not just an extension of the present. In this session, evaluators with foresight expertise will provide an introduction to foresight through scenario work. Specifically, we will engage evaluators in the use of story to use future scenarios to test and examine the assumptions underlying theories of change and future proof them by opening up evaluators’ and our clients’ perspectives and aperture in developing more robust and responsive theories of change and logic models.