Impact Evaluation of Information Literacy initiatives: Case study of EU Erasmus+ projects in the Western Balkans

Abstract

This paper is based onresearch being undertaken in City, University of London. The paper was initially presented online, due to Covid-19 travel restrictions at the ICISIL 2021 Conference, hosted in Brașov, Romania 11th-12th March 2021. The paper describes research conducted to date on this topic of impact evaluation of Information Literacy initiatives in the Western Balkans and describea pilot study which investigates the research question, “Can Contribution Analysis (a Theory Based approach to evaluation) be used to evaluate, the long-term impact of European Union Tempus and Erasmus+ projects effectively?”The research is centred on two projects funded by the EU under the Tempus and Erasmus+ programmes which were conducted in the Western Balkans from 2011-2015 and 2015 to 2018. Both projects were focussed on the theme of Library and Information Literacy development. The universities and institutions involved in these projects were from Albania, Bosnia and Herzegovina, Kosovo and Montenegro. The pilot study itself was conducted in Bosnia and Herzegovina during late 2020 and early 2021.The principal focus of this research is to examine these EU Tempus/Erasmus+ projects and establish if Contribution Analysis, a Theory Based approach developed by John Mayne, can be used to evaluate and measure the impact of the projects on the main participants in the project implementation and their impact also on the wider university community and target groups which these projects were expected to affect.The research has developed a Theory of Change which was used to help build the contribution evidence-based narrative of the impact of the projects post ante.The research has availed of key documents, which are employed at the conceptual stages of Erasmus+/Tempus projects at project application phase, such as the Logical Framework Matrix. The research also utilizes other project documents that are a feature of the projectduring the project implementation, intermediate and final reporting phase. These include project management reports, working group reports,and National ERASMUS+ Office reports, intermediate and final reports. The research has also incorporated partner contributions and data from questionnaires applied in this pilot research in order to link the outcomes, deliverables and causal links of the projects with its intended logic.