Internal Evaluation of Three Unique Projects Funded through the American Rescue Plan Act

Abstract Information: Four American Rescue Plan Act-funded initiatives were outlined by the West Virginia Department of Health and Human Resources (DHHR) for the West Virginia University (WVU) Health Affairs Institute (Health Affairs) to implement with their collaboration and sponsorship, including one initiative that would evaluate the other three. On the surface, the initiatives had very different aims. It was the job of the evaluation team to tell the story of how they all worked toward a common goal of strengthening Home and Community-Based Services for Medicaid beneficiaries. The evaluation team used the RE-AIM framework, taking advantage of its adaptability to pivot toward the needs of the internal teams, the sponsors, and other stakeholders. This work was supported under contract with the West Virginia Department of Health and Human Resources.

Relevance Statement: Four American Rescue Plan Act-funded initiatives were outlined by the West Virginia Department of Health and Human Resources (DHHR) for the West Virginia University (WVU) Health Affairs Institute (Health Affairs) to implement with their collaboration and sponsorship, including one initiative that would evaluate the other three. Three initiative teams worked on the following projects: 1) development of curricula for person-centered, trauma-informed care training and training on conducting assessments required in the West Virginia Statewide Transition Plan, both with a target audience of Home and Community-Based Services direct service professionals; 2) law enforcement officer training around safe interactions with individuals with autism spectrum disorder or other mental or behavioral health disorders, and 3) a public education and outreach effort to educate the community, particularly Medicaid waiver-eligible individuals, around Home and Community-Based Services. All initiatives were evaluated, but a key challenge was keeping in the forefront the singular aim of the funding source: strengthening Home and Community-Based Services for Medicaid beneficiaries. The evaluation team’s adherence to AEA Principles highlights ways this evaluation is unique. One Principle, Competence, was key in understanding how different initiative workflows would be maintained: evaluation team members were embedded in the initiative teams at varying times throughout the project lifecycle; at least one meeting per week was attended by a consistent evaluator. Evaluators were active participants, not only in the evaluation work, but in the implementation of initiatives as well. The evaluation team practiced the Principle of Integrity by ensuring the sponsor and internal teams knew about and approved of this arrangement. The interpersonal domain of the AEA Competencies was present in every aspect of this evaluation. The evaluation began at the same time as each of the initiatives, so change was frequent and significant; this was always communicated, both internally and to sponsors. Communication was key in reflecting the role of the evaluation team and ensuring research efforts among the three initiatives were not duplicated. Transparency around the sharing of findings was encouraged and facilitated. This evaluation, which employed the RE-AIM framework, is relevant to an audience of evaluators because it is likely that large funding streams may be used for differing initiatives more often given the precedence that the environment of the COVID-19 pandemic set in allowing governments and other entities flexibility to determine how funding would be most impactful. The role of internal evaluators could get more complicated with this type of funding structure, and the organization of this evaluation team has lessons-learned to share. This work was supported under contract with the West Virginia Department of Health and Human Resources.