Digital technology for strengthening national evaluation systems

By Khululiwe Faya Manager of the Performance Monitoring and Evaluation Unit of eThekwini Municipality An evaluative culture is about fostering a mindset and nurturing an environment wherein continuous learning and reflection is valued and integrated into the decision-making process of an organisation (whether public or private sector). After attending the sixth edition of the Benin Evaluation Days conference, which took place from 4-6 October 2023, one came to the realization that this continuous learning and reflection occurs at different levels of the decision-making process. Digital technology for strengthening national evaluation systems was the conference's three-day theme. I came away from the conference with a greater understanding of how learning and reflection are essential components of any decision-making process, be it national, international, donor, or M&E sector related. As increasingly diverse panels discussed the introduction /adoption of Artificial Intelligence (AI) and automated data collecting technologies, it became evident to me as an M&E practitioner working in a eThekwini Metropolitan Municipality that my work was not excluded from this significant conversation. AI appeared like a distant future for a practitioner in a local municipality of a developing country. However, it became apparent that AI tools like ChatGPT were already producing data quicker than humans could comprehend. If ChatGPT is not adopted correctly, it could have detrimental effects on evaluators. A panel discussion headed by government representatives started this very difficult conversation which does not seem very far-fetch and too distant in the future anymore. Furthermore, the use of digital technology levels the field in the global evaluation arena in terms of its inevitable efficiency and reliability. During panel discussions, it was mentioned that the Covid-19 Pandemic highlighted the necessity of timely and accurate real-time data collection. This suggested the use of mobile applications, online surveys, analysing large volumes of data, use of data visualisation tools and remote monitoring.   I concluded that this requires a differently trained monitoring and evaluation practitioner in the eThekwini Municipality. Programs for capacity building that teach M&E practitioners how to use technology are important and necessary.  Additionally, eThekwini Municipality needs to have a policy position on the use of ChatGPT and the ethics thereof. A Strathub which is the integrated data source for eThekwini Municipality, must function immediately to allow M&E to start analysing data to inform decision making. Therefore, the integration of digital technology into a national or municipal evaluation system requires a strategic approach. At an organization level, there needs to be policy positions for the adoption or customization of different technologies for M&E use. At a Performance Monitoring and Evaluation Unit level, there needs to be  capacity building initiatives for M&E practitioners on the use of technology geared towards real-time data gathering and analysis. Furthermore, at a personal practitioner level, there must be a willingness to change towards more productive use of technology.

Role of technical and financial partners

The idea of the technical and financial partners in fostering and evaluating culture was something else I took away from the conference. The function of financial and technical partners was described as important  to forming alliances to increase capacity and efficiency of those who are task with evaluating.   With over 15 years of experience in the M&E industry, one is aware of the potential for exploitation associated with outsourcing evaluation studies and the belief held by the private sector that externally conducted evaluations yield better results. Nonetheless, it became evident during the panel discussion of the technical and financial partners that some organisations wanted to work with the government to provide services rather than take advantage of it. The efforts of organizations like the International Initiative for Impact Evaluation (3ie), Twende Mbele, J-Pal Afrique, United Nations Children’s Fund (Unicef), and the Partnership for Economy Policy (PEP) to advance the practice of evaluation in African countries by fostering knowledge exchange, capacity building, and supporting the application of evidence-based decision making impressed me. As a result, I have brought these possible collaborations to the attention of the Chair of the Metro Learning Network, a forum for M&E practitioners from all the metropolitan municipalities in South Africa. Moreover, one looks forward to a dialogue with potential technical and financial partners on how they could assist in capacitating the M&E practitioner within eThekwini Municipality on the use of technology for data collection and analysis. Khululiwe Faya is the Manager of the Performance Monitoring and Evaluation Unit of eThekwini Municipality, She has worked at national and subnational spheres of the Government of the Republic of South Africa for over 15 years.