Mixed Methods Evaluation
Ayesha Boyce, PhD (she/her/hers)
Associate Professor
Arizona State University, United States
Jori Hall, PhD
Professor
University of Illinois at Chicago, Georgia, United States
DONNA MERTENS, PhD (she/her/hers)
Professor Emeritus
Gallaudet University, United States
DONNA MERTENS, PhD (she/her/hers)
Professor Emeritus
Gallaudet University, United States
Location: Grand Ballroom 7
Abstract Information: The power of story is an essential characteristic of transformative culturally responsive mixed methods approaches to evaluation. It is not enough to include the voices of members of marginalized communities. Their voices need to be brought into the evaluation in a systemic way to contribute to the design and conduct of transformative culturally responsive mixed methods evaluations. The transformative and culturally responsive framing of an evaluation calls upon evaluators to address issues of power and include members of marginalized communities so that their stories drive the design of the evaluation. The stories, when combined with quantitative data, need to be made visible in ways that support transformative change in advancing justice. Examples of transformative culturally responsive mixed methods evaluations will be used to illustrate the power of story in communities of color, as well as in international contexts where development programs are shaped to serve the poorest of the poor.
Relevance Statement: The transformative paradigm emerged in response to the voices of members of marginalized and vulnerable communities who argued that evaluation as traditionally conducted misrepresented them, left them worse off, or dealing with interventions that did not adequately meet their needs (McBride, Casillas, and LoPiccolo 2020; Mertens 2020a). As the world faces dramatic challenges, such as the violence against peopole of color and Indigenous communities, climate crisis, a global pandemic, and an increasing gap between rich and poor with the concomitant consequences of these inequities, researchers can ask themselves these questions: How does my evaluation sustain an oppressive status quo? And a parallel question: How can I structure my evaluation so that it leads to a more just world? Critical reflection on answers to these questions can be guided by the assumptions of the transformative paradigm using a culturally responsive lens. The methodological assumption of the transformative paradigm holds that mixed methods are generally needed in order to capture the complexity of the context, understand the need for action, and develop interventions that are culturally responsive and supportive of increased justice. The concept of ethics in the transformative paradigm is expanded to include cultural responsiveness, addressing inequities, giving back to the communities (reciprocity), and providing a foundation for transformative change. Transformative culturally responsive mixed methods can be useful to identify the diversity within the community that need to be included in order to be responsive to the full range of stakeholders and to contribute to increased justice. This means that methodologically, the researcher needs to include time in the design of the evaluation to collect contextual data and to form culturally respectful relationships that demonstrate that they value the dignity and worth of community members. The evaluation community can benefit by learning more about how telling stories as part of a transformative culturally responsive mixed methods study can be of benefit to communities of color and other communities that experience marginalization and oppression. The story telling is important in the early stages of the evaluation as relationships are developed and the context is more fully explored. The stories are important as a way to inform the design of the evaluation in a culturally responsive manner. The stories are important, in combination with quantitative data, to support community capacity to advocate for change based on the evaluation findings.
Presenter: Jori Hall, PhD – University of Illinois at Chicago
Presenter: Ayesha Boyce, PhD (she/her/hers) – Arizona State University
Presenter: DONNA M. MERTENS, PhD (she/her/hers) – Gallaudet University