CHALLENGES IN EVALUATION PRACTICE DURING COVID-19 AND ITS ADAPTIVE METHODS. THE CASE OF MEDIA LITERACY PROJECT IN PERFORMING IMPACT EVALUATION

Abstract

Conducting evaluations during unpredictable times, such as the COVID-19 pandemic, requires additional effort to respond effectively and efficiently to the situation. Apart from standardized approaches and processes, it is important to adjust them properly without compromising the quality of the evaluation. Travel and field mission restrictions during COVID-19 obliged practitioners to transform their ways of working and completely rely on innovative ICT-based solutions. This paper aims to present the modalities of data collection and analysis methods used in performing remote impact evaluation and the importance of relying on ICT-based solutions. The questions this paper aims to address are: Can evaluators rely on ICT, and to what extent? Will it constitute credible evidence of stakeholder representation? Can the impact of context be measured, and how? The recommendations are based on theoretical analysis of the literature and practical examples of the impact evaluation of the Media Literacy Project from remote data collection, evidence synthesis through case studies and its impact, lessons learned presenting the reflection of the evaluator, use of meta-analysis, and the use of evidence-based assessment combined with key informant interviews and targeted literature to produce analysis and make policy recommendations. The paper concludes with recommendations on the model's potential for replication and further development to prevent limitations arising from such an approach.