51 - Longitudinal Evaluation: Building and Maintaining Participant Commitment to Longitudinal Projects

Abstract Information: Great program evaluation considers the stories of program participants in addition to their short- and long-term outcomes. Many processes unfold over time, necessitating longitudinal evaluation designs. Spanning a week, month, year, or decades, longitudinal evaluation poses a host of methodological challenges, foremost of which is participant attrition. Often times evaluators struggle to maintain high response rates during longitudinal evaluations projects, limiting the comparisons and overall conclusions they can draw. The threat of participant attrition and the importance of maintaining participant commitment to a longitudinal evaluation project cannot be emphasized more strongly. Attrition can bias and deteriorate generalizability of findings often leading to erroneous conclusions. While evaluators may be aware of the perils of participant attrition over time, many reluctantly high drop-out and low rates as inevitable. In reality, participant attrition over the course of a longitudinal evaluation project is neither acceptable nor inevitable. This workshop introduces the Tailored Panel Management (TPM) approach to explore how psychological research informs recruitment and retention strategies in longitudinal evaluation projects. Using examples and case studies from more than a decade of research, we will focus on practices regarding compensation, communication, consistency, and credibility that promote sustained commitment to longitudinal evaluation participation. Case in point, we have used the TPM model to garner response-rates in excess of 70% at the end of a 12-year project. This workshop will provide: practical skill-building for real-world evaluation and an in-depth understanding of the TPM approach that participants can apply to current and future longitudinal evaluation projects. Using the Tailored Panel Management approach evaluators can more effectively establish a connection with a broader range of program participants and set up programs for success in the future. Therefore, allowing us to hear from those who are oftentimes less likely to respond.

Relevance Statement: Evaluators strive to rigorously capture and convey the stories and outcomes of program participants. Many times, evaluators propose using longitudinal data collection methods but receive very low response rates which often results in an incomplete picture of what may be happening with the programs they are evaluating. Additionally, low response rates have often been associated with response bias and limit the conclusions that evaluators can draw. Using the Tailored Panel Management approach evaluators can better establish a connection with program participants and more clearly and accurately inform programs for success in the future.