Success stories: a mixed method case study exploring the long-term impact of a global faculty development program for health professions educators.

Abstract Information: Many evaluations of faculty development programs (FDP) on health professions education (HPE) depend on participants’ self-reports, and the measures focus more on shorter term program outcomes. Case studies provide in-depth investigations of long-term program impact, and how the program impact is achieved. Gathering data from multiple sources in a case study can also reduce self-report bias. This paper presents a mixed method case study to examine the long-term impact of FAIMER’s global FDP by eliciting success stories from program participants (Fellows) and stakeholders. Fourteen Fellows from seven sites were recruited as success cases through extreme case sampling. Each Fellow then nominated 10-12 stakeholders, e.g. institutional leaders, colleagues, and collaborators. In-depth interviews were conducted with each Fellow to understand the self-perceived areas of success in HPE, and FAIMER’s contribution to their success. An online survey was sent to stakeholders to assess and verify Fellows’ success. Among the fourteen Fellows, 2 were from Africa, 9 Asia and 3 Latin America. In total, 131 stakeholders (85%) completed the survey. Top reported self-perceived areas of success were career advancement and recognition, leadership development, and contribution to HPE. Similarly, stakeholders also acknowledged Fellows’ success in areas such as leadership, influence in improving education and community health, and research and scholarship. The top contributions of FAIMER’s FDP Fellows reported were expanded professional network, and capacity building in interpersonal development, leadership, and research. Both Fellows and stakeholder provided recommendations to the program. The findings reinforce the continuing value of FAIMER’s FDPs and provide insights for program improvement.

Relevance Statement: The Foundation for Advancement of International Medical Education and Research (FAIMER) strengthens global health professions workforce by offering faculty development programs (FDP) that aim to enhance necessary knowledge and skills related to leadership and management, education methods, project management and evaluation, and educational research and scholarship, with the long-term goal of advancing health professions education (HPE). Program evaluation in faculty development is crucial to help examine if expected program outcomes have been achieved, and how they have been achieved. There is supportive literature indicating that most FDP evaluations tend to focus on short-term effects with an overreliance on self-reports from program participants (Steinert et al., 2016). This limitation is compounded by both recall and social desirability bias. In addition, not everything can be understood quantitatively, particularly in the case of FAIMER faculty development program where there is great variability (geography, country needs, etc.) in the context of the programs and Fellow numbers are small. Qualitative methods offer a more flexible approach and can explain something that numbers alone are not able to reveal, such as identifying the why and how of FAIMER FDP long-term impact. Therefore, we used a case study design with modified (Piggott-Irvine et al., 2009) success case method (SCM) approach for our study. SCM (Brinkerhoff, 2003) is a subset of the more traditional case study method to find out how well some organizational initiative (e.g., a training program, a new work method) is working. SCM analyses the most and least successful groups to identify what worked and didn’t work in the program. However, literature indicated that it difficult to impossible to identify least or unsuccessful program cases. Based on Piggott-Irvine’s approach we explored in-depth only the most successful cases, i.e., successful Fellows. By collecting data from Fellows’ stakeholders, the investigators shifted from solely relying on Fellow self-reports through information validation from multiple stakeholders, who spoke to and verified the success of the Fellow. Using mixed methods also provided methods triangulation adding to the confidence of study findings. To allow programs to stabilize and the long-term impact to show, Fellows eligible to participate in this case study were those who completed their fellowship since the inception of the establishment of the program till 2016. To address the selection bias, a nomination screener was developed by investigators and used by program directors and faculty in nomination of Fellows. The two finalists from each program site were also endorsed by program directors. In the presentation, we will walk through the details of how the multi-country case study was conducted as well as the findings.

References:
Brinkerhoff, R. (2003). The success case method: Find out quickly what's working and what's not. Berrett-Koehler Publishers.
Steinert, Y., Mann, K., Centeno, A., Dolmans, D., Spencer, J., Gelula, M., & Prideaux, D. (2006). A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8. Medical teacher, 28(6), 497-526.
Piggot-Irvine, E., Marshall, S., & Aitken, H. (2009). The three R's of the success case model-Recruitment, response, rigour.