Multiethnic Issues in Evaluation
Giovanni Dazzo, PhD (he/him/his)
Assistant Professor
University of Georgia, United States
Giovanni Dazzo, PhD (he/him/his)
Assistant Professor
University of Georgia, United States
Jori Hall, PhD
Professor
University of Illinois at Chicago, Georgia, United States
Jori Hall, PhD
Professor
University of Illinois at Chicago, Georgia, United States
Jori Hall, PhD
Professor
University of Illinois at Chicago, Georgia, United States
Location: White River Ballroom I
Abstract Information: In this skill-building workshop, we will demonstrate the intricacies of approaching the telling and listening of stories in culturally responsive and sustaining ways. As stories play a pivotal role in the way we, as human beings, narrate our lived experiences every day, there is immense value in how we, as evaluators, request, observe, listen, and share stories. Within evaluation practice, storytelling provides program participants with ways to communicate and critique issues surrounding the policies and programs that affect their everyday lives. As evaluators, we observe, listen, and engage with participants to do justice in the ways we share their stories. However, what happens when we do not trouble our cultural assumptions surrounding the mechanics of what makes a good story? In this session, participants will be asked to consider the following guiding questions as we explore how to improve our evaluative practices: • What happens when we approach the act of asking questions in ways that do not offer participants the space to tell their stories in culturally responsive and sustaining ways? • How can our listening skills be improved if we have primarily been trained to listen and value stories that are told in particular ways, such as the culturally hegemonic view that narratives must be linear and event-based? • If we do not consider the necessity of culturally responsive questioning, sharing, and listening practices, how do our evaluative practices minimize, hide, and further marginalize those who have already been minoritized? • What happens when we encounter non-verbal forms of communication (silence, body language) or refusals to our requests to share stories? How are non-verbal communications or refusals to share stories engaged in ways that do not further minoritize communities? Drawing on our experiences teaching and conducting evaluation in formal and nonformal settings, we will provide a series of tips and a reflexive guide for participants in how they can approach the telling, listening, and sharing of stories in culturally responsive and sustaining ways. Grounding our work in culturally responsive evaluation, critical theories, and critical qualitative methodology, we show how participants can approach the telling and listening of stories as part social science, part art. Through open discussion and individual and group practice, participants will leave the session with a renewed focus on how to employ culturally responsive and sustaining methods—from interviews to focus groups to storytelling (e.g., oral tradition, storyboards).
Relevance Statement: In agreement with this year’s conference theme, we believe storytelling is essential to the discipline of evaluation. Stories not only require us to think about our methods but, in turn, support relationality between evaluators, and among evaluators and communities. Yet, to speak of storytelling without listening loses sight of the requisite skills that come along in the asking and retelling of another individual’s stories. How we listen and retell, as evaluators, often has immense implications for communities, program participants, and evaluation commissioners. And, how we have been taught and socialized to listen requires cultural interrogation, especially if we attempt to work alongside groups who continue to be racialized, classed, gendered, and minoritized in countless ways. In framing this session around culturally responsive evaluation (CRE) principles, we explore the intricacies and value behind the asking, listening, and retelling of stories in ways that are “supported by the cultural appropriateness of epistemology and method” (Hood et al., 2015, p. 304). Based on the questions posed in this year’s conference theme, we reflect and grapple with the power of/behind story: • What ideologies, within evaluation and broader society, sustain narratives that marginalize? • How do our methods perpetuate these injustices, and what must we do to mitigate harm? Thus, we seek to trouble our fundamental assumptions as evaluators when we ask questions or solicit stories from communities. As noted in the guiding questions in our abstract: One assumption is that members of the community will indeed share their stories with us. However, we must be mindful that participants may resist or refuse sharing their stories with us for various reasons. While we may commonly include language around refusing to answer questions within consent forms, we do not sufficiently discuss how evaluators actually handle these moments in practice. Using the terminology of Guillemin and Gillam (2004), this then creates a rift between our procedural ethics (e.g., consent forms) and our ethics-in-practice/everyday ethical issues (i.e., how do we respectfully respond during acts of refusal?). Through activities drawing from our own teaching and practice, we seek to open space that allows us—as evaluators and practitioners, but mostly, as human beings—to explore whether our work can serve as a site for possibility; what Fine and Torre (2004) refer to as the commitment to “review ‘what is’ and ‘what could be’ but also from an ethical belief that critical researchers have an obligation… to help readers/audiences imagine where the spaces for resistance, agency, and possibility lie” (p. 26). Through individual and group activities and discussion, we explore with participants how evaluators can regard the asking, listening, and (re)telling of stories as relational and transformative acts – as sites for possibility that push for mutual understanding and against culture- and value-free approaches couched in rigor.