Research, Technology & Development Evaluation
Jamelle Banks, M.P.H. (she/her/hers)
Health Research Evaluator, Office of Evaluation, Performance, and Reporting
National Institutes of Health
Bethesda, Maryland, United States
Rosanna Ng, M.A. (she/her/hers)
Supervisory Health Science Policy Analyst, Office of Evaluation, Performance, and Reporting
National Institutes of Health
Bethesda, Maryland, United States
Robin Wagner, PhD, M.S.
Director, Office of Program Evaluation, Analysis and Reporting, Division of Clinical Innovation
National Center for Advancing Translational Sciences, National Institutes of Health
Atlanta, Georgia, United States
Location: White River Ballroom E
Abstract Information: A growing body of evidence indicates that diverse, equitable, inclusive, and accessible workplaces lead to higher-performing organizations. U.S. federal agencies, in response to executive mandates, employee concerns, and other motivating factors, have developed DEIA goals and launched a multitude of activities to develop or strengthen their DEIA culture. However, culture change is notoriously difficult to measure. How do agencies know whether they are making progress? Conventional assessment strategies – capturing employee experience through periodic surveys, analyzing hiring practices and metrics, monitoring organization makeup, etc. – provide important but incomplete information about an organization’s DEIA culture. What other strategies might offer additional insights to help leaders and decision-makers create and sustain change? How might these strategies provide actionable data at the individual or organizational level? Which strategies can be carried out frequently, at low costs, and with minimal burden on employees? How might these strategies complement or supplement existing ones to tell a fuller, more nuanced story of an organization’s DEIA progress? In this session, we will explore novel strategies for assessing DEIA culture and discuss how to pilot the most promising ideas. Although this session is organized by a U.S. federal agency, the discussion should be relevant to other U.S. and international work sectors and organizations.
Relevance Statement: *Context* In June 2021, the U.S. President issued the White House Executive Order (EO) on Diversity, Equity, Inclusion, and Accessibility in the Federal Workforce declaring “it is the policy of my Administration to cultivate a workforce that draws from the full diversity of the Nation” (Section 1, https://www.whitehouse.gov/briefing-room/presidential-actions/2021/06/25/executive-order-on-diversity-equity-inclusion-and-accessibility-in-the-federal-workforce/). This EO requires a government-wide DEIA Plan be issued that defines standards of success for DEIA efforts; identifies strategies to advance DEIA and eliminate barriers, where applicable; promotes a data-driven approach to increase transparency and accountability; and supports agency efforts to conduct evaluation, utilizing agency Evaluation Officers and Chief Statistical Officers as well as the requirements of the Foundations of Evidence-Based Policy Making Act of 2018 (Section 3). U.S. executive departments and agencies are required to develop their own DEIA Plans, using data and evidence to assess their DEIA progress (Section 4). The National Institutes of Health (NIH), from which the discussants hail, is part of the U.S. Department of Health and Human Services (HHS). NIH, in recent years, has undertaken a number of DEIA initiatives and activities aimed at NIH staff and the extramural research community it supports, from which the authors have reviewed for examples of DEIA evaluation questions and approaches. These initiatives include NIH-wide UNITE initiative (https://www.nih.gov/ending-structural-racism/unite), NIH Distinguished Scholars Program (https://diversity.nih.gov/programs-partnerships/dsp), Community Engagement Alliance (CEAL) Against COVID-19 Disparities (https://covid19community.nih.gov/), and Transformative Research to Address Health Disparities and Advance Health Equity (https://commonfund.nih.gov/healthdisparitiestransformation), amongst others. Examples from NIH and other federal DEIA activities will be shared for discussion at this session. *Relevance to Evaluation Practices/Methods* Multiple analytic methods are often needed in evaluation studies to understand how initiatives have been implemented and achieved their stated goals. With the complexity of DEIA initiatives aimed at changing organizational culture, it is important to consider how various data sources and analytic methods may be applied in innovative ways to track progress and examine outcomes. Participants in this session will discuss novel methods for using preexisting (e.g., administrative data) and new data sources to establish baseline measures, monitor progress, and test the impact of DEIA interventions on organizational culture.