Collaborative, Participatory & Empowerment Evaluation
Eve Pinsker, PhD
Clinical Assistant Professor
University of Illinois at Chicago School of Public Health
Chicago, Illinois, United States
Eve Pinsker, PhD
Clinical Assistant Professor
University of Illinois at Chicago School of Public Health
Chicago, Illinois, United States
Eve Pinsker, PhD
Clinical Assistant Professor
University of Illinois at Chicago School of Public Health
Chicago, Illinois, United States
Kim Falk, n/a
Evaluation Specialist
Allegheny County Dept of Human Services
Pittsburgh, Pennsylvania, United States
Location: White River Ballroom I
Abstract Information: Evaluation Anthropologists (Copeland-Carson and Butler 2005, Butler 2015), evaluators who apply their anthropological training in ethnography and social theory, have utilized stories and their interpretation in evaluation practice for several decades. We use stories to serve the goal of connecting the lived experience of people who are the "targets" of interventions with evaluation questions about how and whether these interventions have met their goals, while recognizing and respecting the agency and voice of the storytellers. This raises the question of where on the spectrum of participatory, collaborative, and empowerment evaluation approaches our practices are located (Fetterman et al 2014). Answering this question has to do not just with how we collect or gather stories, but who interprets the stories and how, both with respect to individual stories and across a corpus of stories. This question has been thrown into a new light for us as we have learned about the Participatory Narrative Inquiry (PNI) approach to gathering and analyzing stories (Kurtz 2014), which utilizes thematic analysis across stories, as we commonly do as ethnographers. PNI, however, intentionally includes storytellers in the analysis process, and not just simply through member-checking of themes identified by outside analysts who are not themselves the authors of the stories, but through storytellers themselves identifying patterns across the stories, supported both by dialogue and by software designed for this purpose. In this Think Tank session story-based methods used by anthropologists applicable to evaluation practice will be briefly presented, along with comparisons to Cynthia Kurtz' PNI and storytelling as used by systems thinking practitioners (e.g., Bruce McKenzie and Magaly Goirand of besystemics.com). Then we will break into small groups so participants can discuss their own use of stories in evaluation and when it is desirable or possible to use participatory and inclusive thematic analysis practices, before coming back into a large group to discuss our perspectives on when and how we can use a more participatory approach to thematic analysis across a corpus of stories. Thus we will apply aspects of the processes we are discussing, as we will encourage participants to tell their own stories of how they use stories, and our discussion will develop joint themes stemming from our reflections across these stories about story use. We welcome the participation of evaluators who have used story-gathering and interpretation in a variety of contexts and initiatives, or are contemplating how they are planning to use stories and who to involve in the analysis and how.
Relevance Statement: In their evaluation practices, evaluation anthropologists have used stories to highlight what community members and the people we serve think is important to them, including to: • Identify new developments in community, such as the use of new drugs of choice, or identifying unintended but beneficial consequences of an intervention as part of outcome harvesting. • Contextualize data – to highlight the experiences of people who are not in the majority but whose needs are great, such as ethnic groups, immigrants/refugees, LGBTQ, etc. This has been called for in the context of evaluation of public health agencies’ funding of community-based organizations to ameliorate the consequences of the pandemic in marginalized communities. • Highlight what is happening on the ground to key funders including the federal government. For example, in the context of public health funding passed through to community-based organizations (CBOs) to address equities in the burden of COVID, stories giving CBO staff's experience can highlight how their supported activities affect conditions in the lives of people in marginalized communities. Many evaluators utilize stories in similar ways in their practices. In discussions among ethnographically trained anthropologists, we have realized that we do have insights to share from our methods of eliciting stories through interviews, or documenting stories that arise in the context of observations. For instance, in teaching story elicitation for evaluation and research using semi-structured interviewing, we often direct interviewers to start with prompting the interviewee to descriptively narrate a story, before going on to ask questions about its interpretation. This involves the interviewee as co-analyst, after assuring that the descriptive level of data is not distorted by too-early categorization from the evaluator/interviewer. When we learned about Participatory Narrative Inquiry, we were struck by the explicitness of the role of the respondent in not just narrating the story, but interpreting it, as well as the parallels to the way we already conducted co-analysis. Furthermore, we were also struck by the differences – the consequences of being both more explicit and more intentional about supporting the storyteller’s role in interpretation, and what this does to the relationship between “expert” analyst/evaluator and storyteller. PNI has been available for more than a decade, but interest in incorporating it into evaluation is relatively new. Given the growing efforts to address power dynamics through and in evaluation, we want to encourage exploration of the potential of PNI and similar participatory qualitative or mixed-methods analysis approaches for empowerment-based approaches to evaluation. Mary Odell Butler, Evaluation: A Cultural Systems Approach. Routledge 2015. Jacqueline Copeland-Carson and Mary Odell Butler, eds. Creating Evaluation Anthropology: Introducing an Emerging Subfield. (2005) Annals of Practicing Anthropology No. 24. David Fetterman, Liliana Rodrıguez-Campos, Abraham Wandersman, Rita Goldfarb O’Sullivan. Collaborative, Participatory,and Empowerment Evaluation: Building a Strong Conceptual Foundation for Stakeholder Involvement Approaches to Evaluation (2014) American Journal of Evaluation Vol. 35(1) 144-148. Cynthia Kurtz, Working with Stories in Your Community or Organization: Participatory Narrative Inquiry. 3rd ed, 2014. https://www.workingwithstories.org/download.html