When evaluating a programme’s impact it is often important to understand why a particular change happened, and to investigate this in a robust way. Realist evaluation provides valuable insights into how and why programmes lead to change, and can generate transferable lessons to help practitioners roll out or scale up an intervention. However, as yet there are few standards and guidelines governing what counts as a ‘good’ realist evaluation.
In this session, Isabel Vogel, Melanie Punton and Rob Lloyd will reflect on the first year of a three-year realist impact evaluation, examining the Building Capacity to Use Research Evidence (BCURE) programme funded by the UK Department for International Development. The BCURE programme aims to increase the capacity of policy makers to use research more effectively, through building the skills, incentives and systems required to access, appraise and apply evidence in decision making.
A number of challenges have arisen throughout the first year of the evaluation, including:
- How can we identify and prioritise realist context-mechanism-outcome configurations?
- How can we adequately analyse realist data within resource constraints?
- How can we recognise and encompass complexity in our evaluation design?
The presenters will discuss why a realist approach was chosen to evaluate BCURE, reflect on the challenges faced and the lessons learned during the first year of the evaluation, and comment on the potential value realist evaluation can add within international development more broadly.
About the speakers
Melanie Punton is a consultant with Itad, and specialises in qualitative research and theory-based evaluation approaches. She designs, manages and conducts research across a number of themes in Itad, and has experience of a range of innovative qualitative and mixed-methods approaches to impact evaluation. She is applying contribution analysis, realist evaluation and process tracing in several multi-year impact evaluations, including the evaluations of DFID-funded Tilitonse and BCURE programmes.
Rob Lloyd is an Associate Director at Itad, and has expertise in monitoring and evaluation, institutional assessments, capacity development and managing large scale collaborative initiatives.
Isabel Vogel is an independent consultant, specialising in evaluation, research and knowledge management, and learning-based change to strengthen research utilisation and uptake for evidence-based policy and practice, plan for impact in the design of outcomes-oriented programmes, and design and implement evaluation and impact assessment, including theory of change.
Listen again:
Presentation:
Key contacts
