148 - Distilling stories from a complex intervention led complex evaluations

Abstract Information: Development initiatives are becoming more complex each day. Intersectionality between livelihood, health, climate change, multiple marginalities are no longer a new concept. The interventions itself are complex owing to its multi-sectoral, multi-stakeholder and multi-strategy characteristics and are growing at scale. The role of the evaluations or evaluators is no longer restricted to providing summative evidence on key performance measure but are expected to evolve and innovate its methods to distil concrete learnings from a non-linear pathway of change. This presentation expands on the innovative approaches the evaluation used to assess a large-scale government program in Uttar Pradesh, India, home to 243 million people. It tells stories of women who are members of self-help groups (SHGs) - women collectives formed under the Uttar Pradesh State Rural Livelihoods Mission (UPSRLM). The primary objective of this initiative is livelihood promotion. UPSRLM have been implementing a set of predefined strategies for women’s economic enhancement. It seems linear, like most livelihood promotion interventions do. However, this story goes beyond access to finance or economic upgradation. It’s about the aspirations of women in SHGs to move beyond the conventional institution led definition of livelihood (tailoring, horticulture, handiwork businesses) to becoming agents of change inspired by their mentors and forerunners who come from similar rural settings and contexts whom they called Sakhis. Its about the drive towards realizing self-identity and recognition which having a livelihood itself may not be sufficient to provide.

Relevance Statement: The stories are developed as part of a large-scale complex monitoring, learning and evaluation (MLE). The evaluation, beyond the conventional measures of looking at attribution, also focuses on deep diving into emerging learning to inform program strategizing, replication and scale-up. A conscious decision was therefore, made to have a strong learning component, one learning question at a time over 36 months to distil results otherwise not explicitly captured in its theory of change. The MLE design also advocates for the need to blend diagnostic learning to summative evaluation using light touch in-depth inquiries. The same principle is applied to developing these stories from the field using participatory data collection tools. The finding shared in this presentation is one of the various deep dives conducted as part of the evaluation. It seeks to share these learnings and challenges of adopting such a story-based approach in-depth and demonstrate the viability of using stories in their evaluations to evaluators and implementors alike.