Using Evaluation to Communicate the Broader Value of Investment within a Rural Environment

Abstract Information: Traditional economic evaluations that are commonly used to demonstrate program impact and return on investment are often inadequate in telling the full story of programs designed to address broader social and support needs in community settings, especially in environments where there is a long history of institutional and structural challenges. Alternatively, social return on investment (SROI) measures social, environmental, and economic outcomes and primarily uses financial estimates to convey the value and impact of a program. In the Mississippi Delta, a mostly rural area of the southern US, context is important and warrants an innovative evaluation. We used the SROI framework to assign value to intangible program outcomes of the Clinical Community Health Worker Initiative (CCHWI). This presentation will describe how we iteratively developed a context-appropriate evaluation design and will highlight: 1) why the SROI framework is appropriate for the context of this evaluation; 2) how the SROI framework has been adapted and applied to document non-monetizable program components (e.g., altruistic labor, donations) of the CCHWI; and 3) how the findings are being used to illustrate the broader value that the CCHWI brings to the Mississippi Delta region. This work aims to inspire intentional evaluation design that elevates the influence of geographic and historical context on program outcomes. Additionally, we aim to expand on how SROI is useful in measuring program benefits through quantifying the economic and social value a program is likely to create by comparing those benefits to a program’s operating costs over a span of time.

Relevance Statement: A thoughtful evaluator understands there is no “one-size-fits-all” approach to program evaluation and acknowledges that interventions that influence health often have intangible outcomes that are difficult to assign a value to despite having a potential broader significance. In a community like the Mississippi Delta, where there is a long history of institutional and structural challenges (e.g., systemic racism, divestment, and resource deprivation), context is extremely important and warrants innovative and context-appropriate evaluation methods to fully describe a community program like the Clinical Community Health Worker Initiative (CCHWI). A traditional evaluation of investment usually looks at return-on-investment, where the focus is placed on analyzing the cost per patient outcome. The challenge in using this approach is that many community-health programs, such as the CCHWI, are often influenced by hidden institutional factors such as altruism, volunteerism, and community norms, which are not well reflected in estimates of traditional cost-effectiveness and return on investment analyses. Simply calculating a financial return on investment would not fully account for the value of the CCHWI to this community and can underestimate the full scope of program benefits. Therefore, understanding the hidden resources and “non-monetizable” components of the program are important to capture how the CCHWI benefits residents of the Mississippi Delta region. Since the CCHWI is not implemented in a vacuum and operates within an environment where there are many institutional and structural factors affecting the Mississippi Delta region, a flexible evaluation approach that is sensitive to strained implementation capacity and the influence of place is required. Using the Social Return on Investment (SROI) as an evaluation framework helps us explore all aspects of change and expresses them in comparable terms that will help us tell the story of CCHWI and the level of investment required to reduce patient barriers, respond to social needs, and advance health equity within the Delta. It will also allow us to demonstrate broader value to the Delta for both systems and social change. As interest and investment in programs that center social determinants of health have recently increased, evaluators need effective tools to better quantify intangible outcomes that are often generated by public health interventions. This presentation will describe methods used to evaluate the level of investment required to support implementation of a clinical-community health worker model in a rural, historically neglected environment. The application and illustration of this methodology in our analysis will provide an opportunity for attendees to learn how to develop context-appropriate economic evaluations that contextualize systemic challenges, engage program collaborators and participants, demonstrate meaningful outcomes, and produce practice-based evidence.