Learning Together: Monitoring, Evaluating, Reflecting

Monitoring / the regular examination of and reflection on the “gap” between the expected outcome of an intervention and the actual outcome, with activities and agendas being adapted on the basis of this “incremental learning”.

Evaluation / the investigation of the quality, efficiency and relevance of a course of action, measuring its outcomes and impact against a theory of change.

Learning / the process of acquiring new, or modifying existing, knowledge, behaviours, skills, values, or preferences.

“We are not what we know but what we are willing to learn.” (Mary Catherine Bateson)

Why reflect when there is so much to do? In complex settings, such as a protracted conflict, we as practitioners trying to improve the situation must reduce complexity and identify key dynamics. This is challenging, and we often find in hindsight that we could have done better. Our own ability to adapt to the challenges we face is therefore of key importance. One way is to learn from what we did in the past and how well that worked, and by observing current activities and assessing their scope for improvement. Other ways might be more transformative, such as scenario and futures work or a review of organisational theories of change and assumptions of success.

For individuals and organisations working on conflict and peace, the failure to reflect and learn could lead to errors being repeated and opportunities being ignored. Learning relates to us as persons, at the individual level, and as an organisation. It calls for open-mindedness and a readiness for change, and requires time, structures, tools and methods.

“M & E” – monitoring and evaluation – is an essential element of reflection and learning processes and is intrinsic to project management in conflict transformation.

Monitoring refers to the regular examination of and reflection on the “gap” between the expected outcome of an intervention and the actual outcome, with activities and agendas being adapted on the basis of this “incremental learning”. It therefore largely depends on explicit objectives and clear plans showing how they are to be accomplished and reviewed. In conflict settings, projects and programmes must also include an environmental monitoring component to detect any negative impacts of the project on the context, as well as any risks the conflict setting may pose to the project. A conflict-sensitive monitoring system, as well as a conflict transformation monitoring system, would therefore need indicators for the effects, both intended and unintended, and changing risks.

Evaluation is complementary to continuous project monitoring and takes place at various intervals after the implementation of a project or project component. It may be internal (self-evaluation) or external (evaluation by others combined with relevant feedback from / to stakeholders). Often, a mixture of the two is used. Evaluation can be categorised by the desired aims, interaction between evaluator and team (internal, external, joint), or focus/timing. Formative evaluations look at progress to date and recommend improvements, while summative evaluations measure overall achievement, mostly after an intervention. Impact evaluations take place sometime after the intervention and focus on the changes the project produced in the conflict context.

Monitoring and evaluation: results chains and theories of change

Reflection, and especially monitoring and evaluation, relies on clarity. Monitoring and evaluation is aided when assumptions and hypotheses are identified in the planning phase of a project and clearly stated in documents, for example as results chains and indicators. Another popular method is the use of explicit theories of change. This quest for clarity is even more important in polarised settings, where shared understandings cannot be assumed: communication must cross the divides of culture, language and distance.

This leads to a constant questioning of self and partners: do we have a shared understanding of our goals and how we hope to reach them? How helpful explicit hypotheses are for better conflict transformation can be illustrated by the Berghof Foundation’s work on the education system in Bolivia. There, we formulated the following results chain: an activity (e.g. a problem-solving workshop) facilitates outputs (the ability to understand multiple perspectives), which in turn result in outcomes (a change in the way people relate to one another). In the long run, this develops more far-reaching impact (such as a reduction in violence in a polarised community).

Everyone’s perception of reality is limited. That being the case, it is essential to assess the accuracy of any linear hypothesis: “action A results in outcome B”. We must be open to the possibility that other important factors have been missed or ignored. While working in Bolivia, it became clear to the project team that it was necessary to maintain contact with the Ministry of Education, even after the integration of the Peace Culture programme in the Constitution and sectoral law, in order to monitor how the Ministry intended to anchor Peace Culture in its own regulations.

Criteria for assessing activities in conflict transformation and peacebuilding have been set out by the Development Assistance Committee of the Organisation for Economic Co-operation and Development (OECD-DAC). According to these criteria, it is essential to ask “are we doing it / did we do it right?” and to look at efficiency (balancing means and ends) and effectiveness (“did we reach the objectives”?) We should also consider whether the changes effected are likely to be sustainable. An important indicator of success is the assessed impact of the project, i.e. whether the project contributes to goals beyond its sphere of influence. Coherence refers to whether the intervention contributes to or counteracts other interventions. Moreover, it is important that an organisation reflects on the relevance of any activity (“did we do the right thing?”) Reflection on the relevance of an intervention in any given context goes beyond common reflective practice and is thus absent from many monitoring frameworks. There is a danger, particularly in the field of conflict transformation, that practitioners implement projects or programmes, which, despite being exciting, interesting and seemingly conducive to peace, lack the organisational structure or coherence with other projects required for genuine contextual change beyond a limited number of participants.

Beyond monitoring and evaluation: loops of reflection and learning

Learning and change can be based on various levels of reflection. The easiest and most common one is changing actions: we can then try to change the input to get another result. But, and this is the second level, maybe things are not so linear, and our assumptions have been flawed? And, more complex still, how can we as a team or organisation overcome such a blind spot in the future?

Figure 6: Leaning in loops (source: Berghof Foundation, Barbara Unger)

The deepest level of reflection, known as “transformational learning”, is aimed at changing underlying patterns and designing new learning processes. Here, the interest centres less on what the field still has to learn with regard to content – “what to do” – and more on how to create the best possible conditions to learn on different levels and adjust actions accordingly, which is especially important in the field of peacebuilding and conflict transformation. This “learning about learning” is crucial, since even the best efforts at transformative peace work may be ineffective if we fail to learn the lessons available to us. Reflection should cover all elements, such as access, language skills, funding sources, personnel and effective organisational structures: a successful combination of all of these is necessary for effective and sustainable change.

Continuing to improve

One main challenge in practice is that the logic of responding quickly in an ever-changing environment, such as intervening in a violent conflict, is not conducive to simultaneous reflection. It seems that sometimes there needs to be an impulse from the outside, from a person or group specifically tasked with prompting reflection, in order to create the required space in a hectic schedule, and to encourage a shift of emphasis from the practical to the reflective.

An organisational culture conducive to reflection and learning, in the peacebuilding field and elsewhere, entails the allocation of specific time slots, incentives, mechanisms and responsibilities to reflective practice, whilst also recognising the value of ad hoc meetings, even those as informal as a cup of tea with colleagues or an after-work ride home with the project partner. Organisations can benefit greatly from events outside the usual routine, such as retreats or visits from headquarters or external evaluators. Within the field of conflict transformation, more methods of developing an internalised culture of reflection and learning (about failures and successes) must be identified. It goes without saying that the commitment of the leadership in any setting is vital to this development.

Barbara Unger and Beatrix Austin

References and Further Reading

  • Chris Argyris and Donald A. Schoen (1978). Organizational Learning: A Theory of Action Perspective. Reading, MA: Addison-Wesley.
  • Mille Bojer (2018). Transformative Scenarios Process: How stories of the future help to transform conflict in the present. In Berghof Handbook for Conflict Transformation, online edition.
  • Robert Kegan and Lisa Lahey (2009). Immunity to Change. How to Overcome It and Unlock the Potential in Yourself and Your Organization. Cambridge, MA: Harvard Business Press.
  • OECD (2012). Evaluating Peacebuilding Activities in Settings of Conflict and Fragility: Improving Learning for Results. DAC Guidelines and References Series. Paris: OECD.
  • Cheyanne Scharbatke-Church (2011). Evaluating Peacebuilding – Not Yet All It Could Be, in: Beatrix Austin et al. (eds.). Advancing Conflict Transformation. The Berghof Handbook II. Opladen/Farmington Hills: Barbara Budrich Publishers, 459–482.

Online Resources