Inclusive and rigorous peacebuilding evaluation is both vital and complex. In this blog we share examples of how we are innovating our methodologies to move towards participatory and adaptive practice.
The challenges of peacebuilding evaluation
Peacebuilding processes and interventions are enmeshed in systemic relationships with multiple actors in dynamic and potentially volatile contexts. This complexity poses several challenges to Peacebuilding evaluation:
- Fundamentally, there is little consensus on what the desired end state is and how to measure it. The absence of violence or negative peace is not the same as positive peace or the set of structures, institutions, behaviours and attitudes that sustain peaceful societies.
- Conflict contexts do not follow stable trajectories and are subject to unpredictable and regular changes in their causal landscapes. Peace is also a highly political enterprise that introduces causal factors of both local and international origin that are difficult to discern, predict, or measure.
- The stakes are high and the role of evaluation in collecting and using evidence carries risks, which impacts the quality of evidence we can gather.
The pathways through which outcomes are generated by peacebuilding interventions are unpredictable and highly relational.
Innovating with complexity-aware approaches
In response to these challenges, the peacebuilding evaluation field has been innovating with complexity-aware evaluation approaches. These designs favour learning by and through emergent processes of change and with multiple stakeholders, over measuring simple and predefined indicators of success. They centre the relational rather than the technical to fully appreciate how and for whom outcomes emerge at the individual, collective and broader systems levels. In particular, the use of participatory approaches can create and reinforce the collective knowledge of conflict-affected populations with respect to strategies for peace. This strengthens the potential for local stakeholders of varying positionalities to develop and walk along pathways out of conflict, ultimately making them more actionable.
We are part of this exciting moment for the peacebuilding evaluation field. We are a group of research and evaluation practitioners working in diverse contexts and organisations. Together, we are grappling with the opportunities and challenges we face as we innovate our methodologies and operationalise new ways of doing evaluation.
Our inspiration
We take inspiration from the ideas put forward by Robert Chambers in 2015 that reframed rigour as inclusive in order to embrace rather than reduce complexity. His seven canons (eclectic methodological pluralism, improvisation and innovation, adaptive iteration, triangulation, plural perspectives, optimal ignorance and appropriate imprecision, being open alert and inquisitive), stem from a participatory epistemology that underpins inclusive methods. Through this lens we began to make sense of our own M&E practice, through exploring: how we are being attentive to context; appreciating participatory evaluation design as an iterative process; and, looking in to focus on our roles as reflexive facilitators rather than objective judgers.
Applying the canons in our own learning process, we sought to combine and recombine these ideas to our context of peacebuilding M&E. We found further inspiration in Hallie Peskill and Jewlya Lynn’s proposal for rethinking rigor for adaptive programmes, recently expanded by Thomas Aston and colleagues in the context of thinking and working politically. This highlighted how methodological choices influence the credibility and legitimacy of the claims that can be made (often referred to as internal validity). Responding to multiple stakeholders as methodological choices are made surfaces the tension we face when we embrace evaluation and programming as not simply technical endeavours but as primarily political and relational engagements. We experience and navigate these tensions first hand in the contested space which shapes and is shaped by peacebuilding programming.
An emerging integrated framework
Our exploration of these frameworks has resulted in an emerging integrated framework through which we are intentionally learning across the specifics of our experiences, approaching them as experiments in the practice of inclusive rigour. We expect the framework will continue to evolve as we deepen our learning.
Our framework suggests that inclusive rigour in M&E practice becomes operational through three interconnected domains (shown in the centre of the framework).
- The domain of methodological bricolage includes negotiating and making decisions about appropriate design, methodological mixing and choices of specific methods and tools we bring together for the purpose of understanding and evaluating causal pathways – the how of our practice.
- The domain of quality of participation and inclusivity is how we pay attention to the ways in which our processes open up or close down space for different forms of knowledge, particularly of the most marginalised stakeholders (in peacebuilding this might be populations affected by conflict, or stakeholders that are legally or normatively deemed as ‘the enemy’) to be included meaningfully – the who of our practice.
- The domain of utilization is where we strive to respond to different stakeholder needs for evidence and evaluation to inform decision making and achieve the ultimate goal of improved PB programming – the why of our practice.
The intersections between and across the three domains offer fertile ground for exploring both synergies and tensions, where potential levers for shifting practice might be found.
Examples of our framework in practice
Combining and recombining methods in context to ensure critical reflection and optimize learning and adaptation:
The Mali community driven peace programme is implemented through a four-way partnership between Institut Malien de Recherche Action pour la Paix, Interpeace, the Institute of Development Studies and Humanity United. Systemic Action Research is used as a mechanism for generating innovative solutions for community driven peace in Mali. The MEL system for the programme is embedded in the action research design, and includes a combination of real-time documentation of change, linked to iterative cycles of learning, and the use of goal independent evaluation methods such as outcome evidencing within an overarching contribution analysis design.
Ensuring effective participation and inclusion through our methods and processes to stay focused on local demands and being responsive to local needs:
The Images and Indicators project integrated photovoice methods with Everyday Peace Indicators in a participatory indicators process with the aim of amplifying community voices and enhancing the potential for embedding community-level peacebuilding processes. The project explores how combining different forms of participatory methods creates opportunities for multi-faceted learning that attends to research, policy and community priorities. The team proposes that mixed visual and non-visual participatory methods, in this case participatory numbers and photography, complement and amplify each other to embed peacebuilding processes and generate evidence that is complex, empirically robust and transformative.
The South Sudan Cohort 0 (CO) programme is an innovative accompaniment partnership between the Unyoke Foundation, Search for Common Ground, the HiveMind Institute, and Humanity United. It is a multi-year investment in key relationships and networks with the capacity for sustainable responses to recurring system dynamics. Designed around constant cycles of iteration and learning that allow the partners over the long-term to adapt based on lessons that emerge from the programme, C0’s innovative approach to monitoring and evaluation centres on South Sudanese priorities and responsiveness to emergence, and is aimed at building a framework of long-term signs of change by combining adapted outcome harvesting and social network analysis.
Responding to the learning needs of multiple stakeholders within and beyond the project:
The Power and Relationships research initiative in Colombia led by Adapt Peacebuilding in collaboration with Humanity United analyses the implementation of the Colombian peace agreement via municipal peace councils. It focuses upon how horizontal relationships between citizens at a community level and vertical relationships between citizens and state are re-established and leveraged towards peace in the context of unequal power relations. The research compares traditional and participatory peacebuilding methodologies (Systemic Action Research), to ascertain whether more participatory approaches contribute to better peacebuilding outcomes through the differential modulation of relationship patterns and power dynamics. Data is collected and analysed using complexity-aware evaluation methods, including outcome harvesting and process tracing.
Stay tuned for more detail in our upcoming blogs which will share detail of our learning from these examples.
About the authors
Dioma Alamoussa is the Monitoring, Evaluation and Learning Officer at the Institut Malien de Recherche Action pour la Paix
Marina Apgar is Research Fellow at the Institute of Development Studies
Ángela Maria Báez-Silva is Programme Development Manager at Adapt Peacebuilding
Helene Bradburn is the Global Monitoring, Evaluation and Learning Officer at Interpeace
Ayak Chol Deng is a peacebuilder in the 641M network in South Sudan and co-founder of Anataban Arts Initiative
Edwin Cubillos is a human rights activist, photographer and cultural manager and the Photovoice Coordinator at Everyday Peace Indicators
Tiffany Fairey is a Research Fellow in the Department of War Studies at King’s College London and Photovoice Associate at Everyday Peace Indicators
Stephen Gray is the Director of the International NGO Adapt Peacebuilding. Stephen is also a doctoral researcher on questions of power and peacebuilding at the Institute of Development Studies
Livia Rohrbach currently serves as Search for Common Ground’s Regional Design, Monitoring, and Evaluation Officer for the Horn of Africa and the Sudans.
Leslie Wingender is a Director on the Peacebuilding team at Humanity United