167 - Challenges associated with conducting virtual programme evaluations: Narratives from South Africa's M&E practitioners.

Abstract Information: The recent COVID-19 outbreak has compelled the research and development sectors to rethink regular practices and standards to address new challenges and restraints, particularly the restriction to conduct in-person evaluations. While the significance of learning and accountability remains, and the necessity for evidence is critical, M&E practitioners and other development professionals have to embrace technological advancements and use limited donor funds to assess the feasibility of conducting M&E virtually. This article sought to provide insight on the challenges and red blocks associated with conducting virtual M&E in South Africa. The study applied sense-maker to gather stories from 50 M&E practitioners and organisations in South Africa on their virtual M&E experiences and share good practices. The Cognitive Edge’s SenseMaker methodology and software was used to also gather the practitioner’s thoughts and perspectives of each of their stories shared. The article also deployed an online survey (Google Form) to more than 150 M&E practitioners and organisations to complement the stories of the M&E practitioners. The insight generated from the analysis of the stories gathered, revealed, amongst others, difficulties such as digital illiteracy, physical observation limitations, internet and connectivity issues, and inclusion. Finally, the article offers several recommendations to M&E professionals on how to carry out virtual evaluations reliably and improve upon these challenges.

Relevance Statement: In the field of evaluation, evaluators need to share their personal stories and experiences to develop a more comprehensive knowledge of the evaluation process and its developments. There are several reasons why this is essential: Initially, evaluators' experiences can give useful information about the difficulties and possibilities that occur during the review process. Sharing personal experiences and lessons learnt can aid other evaluators in navigating similar circumstances and enhancing the quality of their work. Second, by sharing personal stories, evaluators and stakeholders may create confidence and credibility. Third, evaluators may assist stakeholders in understanding and appreciating the significance of evaluation and their role in ensuring the efficacy of programs and interventions by demonstrating their dedication to the evaluation process and sharing their experiences. The stories of evaluators may personalize the review process and make it more accessible to a larger audience. Evaluation reports and results can sometimes be difficult and technical; nevertheless, sharing personal experiences can assist in humanizing the evaluation process and make it more approachable to stakeholders. Lastly, sharing personal experiences can promote a culture of learning and continual growth in the evaluation sector. By sharing experiences and lessons gained, evaluators may contribute to developing a community of practice dedicated to enhancing the quality and impact of evaluation efforts. Sharing personal experiences is an essential approach for evaluators to contribute to the development and progress of the evaluation profession, create trust and credibility with stakeholders, and encourage a culture of learning and continuous improvement. In addition to adding a new dimension to the evaluation field, the M&E stories can be moulded to cater to various audiences, including funders, policymakers, the media, and the general public. For instance, utilising "success stories" can assist in communicating and sharing best practices with other M&E practitioners. Furthermore, they are an efficient method for discussing and exchanging effective solutions to problems. The study contributes to answering the following issues: 1. How has the COVID-19 epidemic altered the monitoring and evaluation practice in South Africa? 2. Which virtual procedures and tools have been implemented by organizations and practitioners to respond to the problems and possibilities presented by the virtual monitoring and evaluation process? 3. In what ways have the recently implemented methods of monitoring and evaluation shown to be successful?