Diversifying the STEM Workforce in Philadelphia: The Use and Value of Stories as an Ecologically valid approach to program evaluation

Abstract Information: Despite efforts to diversify the US STEM workforce, Black and Hispanic individuals remain underrepresented. The root cause of this is multifaceted and perpetuated by inequities in educational opportunities. For example, schools serving predominantly white students are more likely to offer Chemistry and have more effective teachers than schools serving majority Black and Hispanic students (CRDC, 2014). Entrance into the STEM workforce is difficult without access to gatekeeper courses. In this session, we share evaluation stories from two programs in Philadelphia, FirstHand and BULB, that aim to diversify the STEM workforce by supporting participants’ development of STEM competencies and interest and demystifying the process of entering the STEM workforce. Despite being a haven of STEM innovation (Vey et al., 2017), Philadelphia classrooms are underfunded and lack resources to prepare students to join the STEM workforce. This is compounded by the lack of job opportunities in STEM for those without at least a 4-year degree. To bridge this innovation ecosystem with local talent, FirstHand partners with local schools to provide middle schoolers with hands-on, project-based, out-of-school STEM lessons. Similarly, BULB is a 4-week apprenticeship-style, workforce development program that trains and connects predominantly Black Philadelphians experiencing low incomes to jobs and mentorship with local STEM companies. As evaluators of these programs, we will share how we have used stories to inform design and measure outcomes. We aim to highlight the essential role stories play in evaluation work and to promote the use of stories as a valuable, ecologically valid approach to program evaluation.

Relevance Statement: Despite concerted efforts, Black and Hispanic individuals remain underrepresented in the STEM workforce (Nadeem, 2021). Increasing diversity in the STEM workforce has the potential to decrease economic gaps that span racial, ethnic, and gender lines. But more importantly, STEM is the foundation of innovation, and diversity adds new perspectives that lead to impactful discoveries. Unfortunately, Black and Hispanic individuals often face educational hurdles that prevent their preparation for and entrance into STEM fields, such as (1) a lack of access to advanced math and science courses, (2) less qualified teachers, and (3) lack of opportunities to enter the STEM workforce without a 4-year degree. In this session, we share stories from two programs each taking a converging approach to diversifying the STEM workforce, FirstHand and BULB (Building an Understanding of Laboratory Basics). FirstHand and BULB are programs in Philadelphia that aim to diversify the STEM workforce. They are both based out of the University City Science Center, the oldest and largest urban research park in the United States (Puckett, 2022). Both programs serve as a bridge between the underresourced broader Philadelphia community and Philadelphia’s rich STEM ecosystem within the Science Center which includes business incubation, healthcare innovation, and start-up companies (Global Philadelphia Association, 2023). FirstHand bridges these two communities by providing free, yearlong science programming to middle and high-school-aged students. Similarly, BULB, an apprenticeship program that offers lab training and access to science and technology companies, aims to diversify the STEM workforce by serving predominantly Black Philadelphians coming from low-income backgrounds. AnLar serves as external evaluators who aim to understand the key components of these two programs as well as provide evidence of their impact. While we value using reliable and validated measures for program evaluations as illustrated by our use of measures such as the STEM Program Quality Assessment developed by the Weikart Center at the Forum for Youth Investment and The Panorama Student Survey (Smith et al., 2012) (Panorama Education, 2020), these measures can be impersonal and disconnected from the aim and context of the programs. Evaluations that only use these measures also lack the perspective of the participants, designers, facilitators, and other stakeholders who use and benefit from the program. Our evaluation design for both programs includes collecting and analyzing the stories that the program facilitators, industry partners, and participants tell and leveraging them to both inform evaluation design and as outcome measures. Collecting and listening to these stories has been critical to promoting stakeholder engagement (Reid, 2020) and has guided the design of evaluations of both these programs with support from the U.S. Department of Education's Education Innovation and Research (EIR) Program and the City of Philadelphia Department of Commerce. Our goal in this session is to share our approach to using stories to both inform evaluation and as an outcome of the evaluation to determine impact. We aim to highlight the essential role stories play in evaluation and to promote the use of stories as a valuable and ecologically valid approach to program evaluation.