Behavioral Health
Crystal Luce (she/her/hers)
student
University of Colorado - Denver, Colorado, United States
Crystal Luce (she/her/hers)
student
University of Colorado - Denver, Colorado, United States
Brian Dates, M.A. (he/him/his)
Lecturer - Public Administration and Policy
University of Michigan-Dearborn
Beverly Hills, Michigan, United States
Brian Dates, M.A. (he/him/his)
Lecturer - Public Administration and Policy
University of Michigan-Dearborn
Beverly Hills, Michigan, United States
Rachel Carlson, n/a
Research Assistant
Aurora Research Institute, United States
Rachel Carlson, n/a
Research Assistant
Aurora Research Institute, United States
Antonio Olmos, Ph.D.
Executive director
Aurora Research Institute, United States
Antonio Olmos, Ph.D.
Executive director
Aurora Research Institute, United States
Location: Room 301
Abstract Information: The Substance Abuse and Mental Health Services Administration (SAMHSA) federal agency requires that grantees receiving funds collect an outcome instrument known as National Outcomes Measures (NOMS). In the last year, SAMHSA changed the instrument in ways that may be good for programs and clients but not necessarily good for demonstrating effectiveness. In this presentation we will present the changes in the NOMS, and how evaluators who have been collecting and analyzing NOMS data for several years, are transitioning to a new way to train data collectors, collect, and analyze the new NOMS. The panel will present three stories from three very different groups: One story is related to the collection of a program serving youth (0-24 y.o.) asylees, immigrants, and refugees, and the impacts with data collection. The second story is related to the implementation of the changes associated with a specific type of funding named Certified Community Behavioral Health Centers (CCBHC), in particular, the re-training associated with a second round of funding for the same program, and the changes associated with changes in the development of the sample, data collection, data entry, and some of the tips and tricks we developed for data tracking and submission. Finally, the third story is related to the use of the NOMS as outcome measures for programs and a description of the effectiveness of the interventions with normative information for sex, ethnicity subpopulations, including SPSS syntax used to transform and analyze the data.
Relevance Statement: As a national tool, the National Outcomes Measures (NOMS) can be beneficial for tracking program outcomes and potential comparisons to similar programs. In December 2022, SAMHSA began utilizing a new iteration of the NOMS. SAMHSA discussed the benefits of the tool being shorter and potentially easier to use. Yet, the rollout of the new measure has not been uneventful for both the funding agency, and the local program evaluators in charge of data collection and analysis. The three presentations in this panel relate the ways in which evaluators who have collected and analyzed NOMS data for several years, have transitioned from the old NOMS to the new NOMS. Each story will add experiences and lessons learned by every group to the general knowledge of those who may be experiencing NOMS for the first time, or those who may have many years of experience in the collection of NOMS in the field but have not had a direct experience with the new NOMS. The three presentations will add to the knowledge and skill set of evaluators in different ways: The first presentation will address the effects related to the changes in the item scaling from Likert to dichotomous (YES/NO) scales. A big concern is how to establish changes in the participants, when the instruments have changed to the point where straight, pre-post comparisons may not be possible anymore when the participants completed the two different versions of the NOMS. The same presentation will describe reflections about how to make the instrument culturally responsive to the multitude of people and cultures that the program served. The second presentation will describe the approaches to data collection and analysis in terms of the use of the new instrument to assess effectiveness of the interventions with normative information for sex/gender identification, race/ethnicity subpopulations. This program will also add knowledge and skills by providing examples of SPSS syntax files intended to manipulate and conduct analysis based on the previous parameters. The third presentation will talk about the retraining associated with the new implementation of the NOMS. This presentation will add knowledge and skills through the sharing of tips and tricks associated with the development of the new requirement for some grants to collect a random sample of participants in the grant, as well as tricks related to data tracking and data entry (such as the use of bots) into the SPARS portal.