117 - Whose Stories Get Told? Toward Validation and Representation in Culture and Climate Studies

Abstract Information: Culture and climate studies are a common practice in program evaluation. They are an important component in understanding the varied experiences and uplifting stories of program participants. During the course of evaluating multiple large National Science Foundation-funded Centers, our evaluation team encountered multiple challenges with development, implementation, and analysis of a mixed method (surveys, observations, interviews, and focus groups) culture and climate study. In this roundtable, we will discuss and contemplate the following with the audience: a) How can evaluators increase participation rates in culture and climate studies, with the acknowledgement that the data collection and reporting of sensitive information has potential consequences for respondents? b) How can cognitive interviews contribute to the initial validation of culture and climate survey instruments? c) How should evaluators analyze quantitative data in order to construct valuable interview and focus group protocols to add depth to study findings and produce useful and actionable feedback for program managers? In this roundtable, we hope to brainstorm with others on how to improve the culture and climate study process.

Relevance Statement: It is a grand challenge to recruit, train, and encourage students and researchers to pursue advanced degrees and careers in Science Technology Engineering and Mathematics (STEM). From improving quality of life by enhanced individual economic opportunity and increased advances in healthcare technologies, to increasing the vitality and creativity of STEM projects (Greene, DeStefano, Burgon & Hall, 2006) and continuing to be a player in the global economy (National Science Board, 2008), these efforts continue to gain traction. The benefits of such endeavors are incredibly high no matter whose motivations are at stake. As such, the evaluations of these programs continue to increase in importance. Culture and climate studies offer evaluators an important means to understand participant stories related to values of equity and inclusion.