Proving Promise and Support: Evaluation of Indiana Family Preservation Services

Abstract Information: In June 2020, Indiana Department of Child Services (DCS) implemented a new service, Indiana Family Preservation Services (INFPS), which aimed at keeping children and families intact when it was safe to do so. Evaluating INFPS required creating intentional partnerships with internal and external stakeholders to understand how the program was operating and what program areas needed improvement. The evaluation was a process of continuous quality improvement that required listening to the stories of each stakeholder involved in the implementation of INFPS. In this session, we will focus on the following areas: 1) Implementation of INFPS; 2) Survey and interviews conducted with DCS caseworkers; and, 3) Evaluation of INFPS and Implications for Policy & Practice. We will address barriers and strengths of the program from each perspective: program implementation, interviews with caseworkers, and the formal evaluation. The ultimate goal of each area of research is to understand whether INFPS is able to effectively keep children safely with their families by reducing the number of child removals and the amount of repeat maltreatment in Indiana. We will discuss opportunities for program improvement and future directions of INFPS research.

Relevance Statement: The Family First Preservation Services Act (FFPSA) challenged states to adopt programs to help keep children in home when it was deemed safe to do so; this presentation highlights evaluation opportunities and barriers encountered while studying Indiana Family Preservation Services (INFPS). We will focus on the story of how we arrived at our data. This process involved working closely with internal and external stakeholders to identify how the program operated prior to evaluating the program. Yet, after the evaluation began, it required a team of research analysts to embark on a journey of continuous quality improvement as we worked with all stakeholders to gather the data needed to evaluate INFPS. Prior to the analysis, we utilized contribution analysis. Contribution analysis is a method not commonly used in child welfare research; however, contribution analysis addresses many limitations in past intensive family preservation program (IFPS) evaluations. Prior research on IFPS programs has been complicated by research sampling (Fraser et al., 1997), lack of comparable control groups (Rossi, 1992), missing data (Orsi et al., 2012), and the inability to parse out fidelity to the treatment models (Schweitzer et al., 2015).Through our intensive monitoring of INFPS, we were able to identify and react to barriers as they presented themselves. Contribution analysis provided a critical first step into evaluating the program and helped mitigate other potential confounding factors that could have influenced outcomes. We believe that contribution analysis is an important tool that researchers/evaluators should consider in the process of program evaluation as it provides a tool to monitor whether the program is being used to fidelity or not. At the same time, we were able to listen to the stories of DCS case managers who actively used the service with families to understand their perception of how the program was operating and what areas they believed to be barriers. Throughout this evaluation, understanding how the program operated in real time was vital to the evaluation. Learning from practitioners/professionals serves a valuable part of evaluation to identify barriers and strengths of programs being evaluated. Finally, we utilized a quasi-experimental research design, which consisted of a treatment and control group. The treatment cohort consisted of all children and families who started INFPS services between January 1, 2021 and March 31, 2021. The control group consisted of children and families who started as an in-home CHINS or IA between January 1, 2019 and March 31, 2019. We will address the barriers that exist in child welfare research when it comes to evaluating programs and services, and the implications it has on child well-being. Furthermore, we will address how research design has impacts disproportionality within child welfare.