Social Impact Measurement
Slavica Stevanovic, MBA, MA, CE (she/her/hers)
Managing Partner
Qatalyst Research Group, British Columbia, Canada
Location: Room 309/310
Abstract Information: “The truth about stories is that that’s all we are”, Thomas Kind tells us in his book ‘The Truth About Stories – A Native Narrative’. Stories shape who we are, where we came from, and how we understand and interact with others. Qualitative data or storytelling has become an accepted methodology in decolonized and Indigenous-based approaches for evaluating the effectiveness of programs. However, there continues to be some resistance to incorporating stories in the emerging field of impact measurement and measurement. Investors, particularly private sector investors, tend to prioritize the use of quantitative data and the efficiency of data collection. Using personal stories to demonstrate or illustrate impacts is often considered time-consuming, expensive, and not representative. It is also messy. How would we choose whose story to tell? Do we only tell positive stories? What happens to other stories? It is important to have strong quantitative data. We need to collect extensive, representative and defensible data on impacts (e.g., data on financial returns, scalability, employment, gender balance, health outcomes, emissions reductions, technology development, etc.). However, we must not forget the stories that numbers cannot tell. This presentation will demonstrate how stories draw us in, bring impacts to life, engage our emotions, improve understanding, create empathy, and even raise funding. The stories can also decieve us and create illusions of success. We will use examples from multiple projects to illustrate how storytelling identified challenges, gaps, and unintended outcomes, painted a more complete picture of the impacts, and contributed to significant improvements in future investments.
Relevance Statement: Stories are a powerful tool. They are often used very effectively in evaluation to support quantitative findings, illustrate qualitative findings, and add color to the reported impacts. Many evaluation reports incorporate personal stories, as well as case studies, quotes, and testimonials, to support the quantitative or survey findings and engage readers. However, stories are used much less extensively in the field of impact measurement and management. As an IMM tool, they are often discounted as not representative and lacking in value. Even when they are used (mostly as an anecdotal marketing or communication tool), they are often not well researched or factored into the analysis. Our research and analysis, supported by our experience in conducting impact measurement major public and private investments studies over the past 15 years, indicates that, when well designed, storytelling and other qualitative analysis not only strengthen communication of the findings but also deepen our understanding of the impacts and the beneficiaries, challenges, gaps, unintended outcomes associated with the investments, and the opportunities to improve future investments. Drawing from a series of IMM projects, in this presentation, we will demonstrate that stories should be an integral part of any impact assessment and not only a ‘nice to have’ so investors can feel good about their investments. The project examples we will use for this presentation will demonstrate how stories and other qualitative data collected from the communities and external stakeholders were used to highlight vital impacts, issues and challenges and improve future decision-making. Examples will be drawn from public and private investments involving impacts ranging from technology development and adoption, women and minority-owned enterprises, economic development to the environment, homelessness and housing. We will discuss ethical issues that need to be considered in inclusion/exclusion criteria when deciding whose story to tell and how to tell it in the most effective way so that investors and decision-makers can take action to improve their investment theses, criteria, selection, program delivery, and reporting.