A Story of Victory and Challenge in Strides Toward Equitable Success: What We Learned from Our National Evaluation of an Initiative Supporting Black, Indigenous, and Latina/o/e STEM Doctoral Students

Abstract Information: Our team conducted a rigorous evaluation—grounded in equity and cultural responsiveness—to understand the efficacy and impact of a decade-long, foundation-funded initiative housed at eight universities. Programs provide financial support, mentoring, professional development (PD), and other resources to promote recruitment, retention, and completion in Science, Technology, Engineering, and Mathematics (STEM) among Black, Indigenous, and Latina/o/e (BIL) doctoral students.

Panelists will share a collection of Key Learnings gleaned from this work related to (1) student outcomes; (2) program design and implementation; (3) student experiences; (4) supporting BIL students; and (5) stakeholders’ perceptions of impact and institutionalization. Highlighted below is a summary of Key Learnings, which will be further elaborated and contextualized during the session:

Student Outcomes:

The program realized tremendous student success—93.4% of students have been retained (i.e., still enrolled or graduated). More than 72% of students graduate within seven years, compared to 42% of BIL students nationally.

Program Design and Implementation:

Recruitment: Promising recruitment strategies include pipeline programs intended to prepare and encourage racially minoritized undergraduate students to pursue graduate education; participation in conferences that attract BIL undergraduates; hosting of undergraduate research programs; and engaging students in recruitment.

Student Support: The most salient student supports were mentorship, financial support, professional development, and community engagement. The evaluation revealed the best support makes it easy for students to get the help they need, is based on a holistic view of student success, and accounts for the impacts of systemic racism on student experience.

Student Experiences:

Mentorship: Students valued mentorship, but experienced a gap between what they received and what they desired. Students envisioned exemplar mentorship as being: networked, holistic, and culturally responsive; personalized and one-on-one, with regular interactions to build trust/connection; and involving mentors with similar lived experiences and backgrounds.

Financial Support: Scholarships and other financial resources were described as one of the greatest program benefits, offering students flexibility and increased access to opportunities. Funding was attributed to making the PhD attainable.

PD: Students appreciated the holistic manner with which program structures helped to facilitate their personal and professional development. Early career focus helps build the skills needed for their desired careers.

Community: Some students felt that their academic departments didn’t explicitly value DEI and therefore didn’t serve as inclusive environments, making the program the only space where students felt welcomed and valued.

Supporting BIL Students:

Students recounted how minoritized learners within STEM encounter differential experiences, requiring tailored support structures to aid in facilitating their success. Students described a robust ecosystem of support necessary for their success comprising three domains of influence: Individual, Programs and Networks, and Systems and Cultures.

Perceptions of Impact and Institutionalization:

Stakeholders across campuses shared that program engagement led to: creating a community of learning and support among institutions; serving as models for recruitment and retention; shifting toward holistic graduate admissions processes; and raising visibility about departmental climate and culture challenges.

The session will conclude with strategic recommendations for funding or implementing similarly-missioned initiatives aimed at facilitating equitable BIL student success, including cultivating inclusive environments and addressing systemic racism.

Relevance Statement: Our team employed a concurrent, triangulation mixed methods design to understand the efficacy and impact of this decade-long, multi-institutional equity-focused STEM initiative. During our engagement in this evaluation, we demonstrated the importance of rigorous methodological techniques, which of course included technical competency, but perhaps most importantly, collaborative, culturally responsive, and asset-based approaches. A collaborative evaluation orientation was integral to shaping the study’s design. For example, we initiated the evaluation with a series of conversations with diverse program stakeholders (e.g., foundation staff, advisory members, program teams, students and alumni) to solidify assessment priorities, identify core program components, receive input on the evaluation plan’s appropriateness and feasibility, and discuss evaluation expectations—including for engagement, dissemination, and utilization. Relatedly, our team engaged member-checking processes, wherein program stakeholders were afforded the opportunity to review and offer feedback on developing understandings and artifacts as part of our sense-making process.

Moreover, culturally responsive and assess-based approaches took many forms, including our value in and leveraging of diverse ways of knowing. In addition to integrating various qualitative and quantitative methods, we intentionally centered the voices and experiences of program stakeholders (e.g., team members, students, administrative partners, institutional leaders), who we acknowledged as keepers of knowledge and experts on many fronts, and especially with regard to their lived experiences. This was particularly instrumental when considering students, who shared having to navigate campuses and departments where they don’t feel heard, seen, or understood. Our evaluation worked to actively disrupt that norm.

The relevance of this work is further demonstrated by the articulated value and utility to program stakeholders, and most notably, the initiative’s funder. Learnings gleaned from this evaluation encouraged the Foundation to rethink the program model, with an emphasis on systemic change that reveals and addresses biases and barriers deeply entrenched within university systems to truly address the root of STEM underrepresentation. They expressed a deep appreciation for how the evaluation’s design foregrounded student voices to inform a larger discussion of what it means to support minoritized doctoral students in STEM. This assessment informed the Foundation’s reimagining of program design and initiative success—they announced a newly envisioned program in March 2023, building on the most effective components of its predecessor and addressing previously observed gaps. This evaluation, and the Foundation’s application of its learnings, can serve as a model for other funders about what it means to fully invest in and meaningfully utilize program evaluation. For evaluators, it could offer an illustration of what collaboration and thought-partnership might look like in these contexts.

Ultimately, this evaluation revealed rich insights into what program leadership, students, and institutional stakeholders believe contribute to a successful initiative. Our team had the benefit of considering these insights, informing our development of 17 strategic recommendations related to program design and grant administration. Collectively, understandings gleaned from this evaluation effort can be applied by evaluators (in terms of developing and executing a quality assessment of this type), as well as funders and practitioners as they endeavor to improve and transform graduate STEM education for BIL students.