Mixed Methods Evaluation
Taylor Anderson, MS (she/her/hers)
Senior Associate
Public Profit
Oakland, California, United States
Sylvia Kwon, PhD (she/her/hers)
Senior Consultant
Public Profit
Location: Room 203
Abstract Information: Surveys are a tried-and-true way to get feedback, but unfortunately, many mission-driven organizations get buried under a bottomless pile of survey data that may leave out the perspectives of younger people or those who prefer to communicate verbally or visually. Plus, many people are tired of taking surveys! Broadening how you solicit feedback is more inclusive and can lead to more robust information to improve program quality for all participants. In this session, attendees will learn about in-person and virtual data collection methods to ensure a broader representation of audiences. Using Public Profit’s Creative Ways to Solicit Feedback guide, attendees will get hands-on practice using verbal, kinesthetic, and visual methods that will grow their toolbox of non-traditional methods for gathering feedback, equitably and inclusively. This skill-building session seeks to spark motivation and a newly found sense of enthusiasm for data collection!
Relevance Statement: Many mission-driven organizations get buried under a seemingly bottomless pile of surveys. Different funders and partners commonly require separate surveys, leading to widespread survey fatigue for participants and staff. Moreover, surveys aren’t always the best fit for very young respondents, those who speak a language other than English, and those who feel more comfortable sharing input verbally. To have more equitable and inclusive evaluations, it is essential to solicit feedback from a wide range of people, not just those most comfortable filling out a survey or speaking up in a group setting. Participants in this session will learn and practice non-traditional ways to solicit feedback, growing their toolbox of data collection strategies with verbal, kinesthetic, and visual methods. These approaches are useful for collecting a variety of inputs (both in-person and virtually), approaching those who are over-surveyed, and working in settings where traditional approaches have historically been inequitable or excluded people from voicing their opinions. Participants will walk away from this session with data collection approaches that will help to assure that everyone – regardless of race/ethnicity, age, ability, formal education, or language fluency – has an opportunity to share their experiences and voice their opinions. Examples of data collection strategies that will be shared include Photo Responses, Personal Meaning Maps, and Feedback Walls from our publicly available Creative Ways to Solicit Feedback guide. Variations for in-person and virtual offerings will be described, and attendees will be able to practice select methods during the workshop. The guide includes step-by-step instructions on these activities (and others) so that evaluators can incorporate and adapt Creative Ways into their practice. A link to the guide will be included in the session material for participants. Ideally, feedback should elicit the lived experiences of all participants, offer alternative ways to share sensitive information, and build toward a common understanding of problems and solutions among the group. The practical strategies shared in this session are designed to deepen how programs and organizations engage with and learn from participants, an essential and critical step in promoting culturally responsive and equitable evaluation and practice. Throughout the session, activities and examples will be grounded in the guiding principles of the evaluation and learning framework developed by the Equitable Evaluation Initiative: - Evaluation and evaluative work should be in service of equity. - Evaluative work can and should answer critical questions about the ways that a strategy impacts different populations and addresses systemic drivers of inequity. - Evaluative work should be designed and implemented commensurate with the values underlying equity work – that it is multiculturally valid and oriented toward participant ownership.