Evaluating Progress in 988/Crisis System Implementation: Metrics and Recommendations

Abstract Information: In July 2020, the Federal Communications Commission (FCC) established 988 as the new designated national number for suicide prevention and mental health crisis. It requires that all telecommunication providers direct 988 calls to the National Suicide Prevention Lifeline’s network of approximately 170 crisis centers. The easy-to-remember 988 number is designed to quickly provide Americans experiencing a mental health crisis with immediate access to care. For successful implementation, 988 must go beyond the operations of call centers, incorporating changes in mobile crisis and crisis stabilization as well. Further, to be effective, the system will need to be supported by infrastructure development and changes in the policies and protocols of crisis service-related systems. As we invest in this new national system of crisis care, how can we move beyond basic monitoring metrics of the 988 crisis line to tell a broader, more unified story of national progress?
Established in 2010, the National Action Alliance for Suicide Prevention (Action Alliance) is a public-private partnership working with over 250 partners to advance the National Strategy for Suicide Prevention. Because of its unique composition stakeholders and its reputation as the nation’s leader in suicide prevention, the Action Alliance is well-positioned to explore a more complete story of 988/crisis implementation through broader measurement of national progress. In this paper, we describe how the National Action Alliance, supported by a panel of crisis system experts, created Assessing Progress in 988/Crisis System Implementation: Metrics and Recommendations, a guide to creating a unified evaluation of the 988/crisis system.

Relevance Statement: Moving beyond traditional process monitoring metrics for 988/crisis services (e.g., call volume, answer rates), this work uses best practices in systems change evaluations, beginning with the complex task of first concretely defining the boundaries of a broad, nebulous, national 988/crisis system and then articulating key players and their interconnection. Based on this work, the Action Alliance identified and convened a panel of experts in the 988/crisis system space. Working together with the expert panel, the Action Alliance articulated a clear set of objectives for measuring national progress in 988/crisis system change and created a complex logic model, reflecting the changes required to produce the short-term, intermediate, and long-term outcomes reflective of successful 988/crisis services functioning.
Continuing to work with the identified panel of experts, the Action Alliance then explored the landscape of existing metrics related to 988/crisis services across a variety of fields and systems, identifying issues of relevance, validity, reliability, timeliness, and quality. Supported by an understanding of existing metrics, the team then explored gaps in metrics and produced recommendations. Resulting recommendations to further our understanding of the 988/crisis system include improved coordination of public and private sectors and their measurement activities related to 988/crisis services, public sharing of progress markers in 988/crisis system infrastructure building, and the development of shared metrics to support an understanding of equity issues in 988/crisis system roll-out. The Action Alliance has used this work as a catalyst for bringing together new partners and exploring recommendations on how a comprehensive evaluation of 988/crisis service implementation could be advanced.
The presentation will provide key lessons learned related to the importance of concretely defined boundaries in evaluation and the benefits of engaging a well-informed and well-connected expert panel. The audience will be able to explore our process for identifying and rating existing metrics, which could prove beneficial in broad-based and systems evaluations. Finally, AEA’s audience will connect with the real-world application of a proposed national-level evaluation that is working with a timely, well-publicized issue in public and behavioral health.