Since its inception in 2015, the United Nations' Sustainable Development Goals (SDGs) have captured global attention, offering a bold vision for a better future. With the ambitious deadline of achieving these goals by 2030, the question that looms large is this: How do we know if the United Nations (UN) is making progress towards these goals? The answer lies in evaluation—a systematic assessment of institutional performance. Expert teams gather data on implemented activities, engaging with stakeholders and affected populations, to measure the extent of achieved objectives and their real-world impacts. This process culminates in lengthy reports, brimming with evidence-backed insights and recommendations that can guide crucial decision-making. These reports are then used by member states, management of international organizations (IO), implementing partners, affected populations, researchers, and others.
Unsurprisingly, evaluation has flourished in IOs. Thousands of consultants work with UN evaluators and produce hundreds of evaluation reports annually. The financial commitment to this cause is staggering, with an estimated spending of approximately USD 430 million each year—an amount equivalent to the entire annual budgets of major organizations like UNESCO or the International Labour Organization.
However, evaluation is not a value-free, purely technocratic endeavor. Our new book, The Politics of Evaluation in International Organizations (Oxford University Press, 2023) shows that to fully unleash the potential of evaluation, we must confront the undeniable political nature that underpins this vital process. By acknowledging the political dimensions at play, we can deliberately design evaluation systems to minimize political influences, thereby enhancing their independence.
Evaluation is (also) political
Evaluation can be useful for institutional learning, course-corrections, and accountability. Indeed, many view it as a functional tool in the policy process, assisting in the achievement of development goals. But the other side of the coin—oftentimes conveniently neglected—is the political use of evaluation. Evaluation does not exist in a vacuum—but is surrounded by actors with different political interests who seek to influence evaluation results and use. Based on 120 research interviews with evaluators, diplomats, and IO secretariat officials, The Politics of Evaluation in International Organizations provides empirical evidence as to how actors use evaluation to justify their pre-defined positions on policy decisions, thereby politicizing the evaluation instrument.
For example, powerful states tend to cherry-pick evaluation results and use them as a bargaining chip in intergovernmental negotiations or to contain the management’s bureaucratic influence. IO administrations also use evaluation strategically to increase their bureaucratic influence vis-à-vis member states or to exercise control internally.
Evaluation reports contain political biases
Stakeholders can also affect the content of evaluation reports. For instance, we found systematic differences as to how evaluation recommendations are written, depending on which stakeholder controls evaluation system resources. In UN organizations where member states control evaluation system resources, such as staff appointments, budget, and agenda, recommendations are more specific in their language, leaving less room for interpretation as to whether or how the recommendations should be implemented. Such recommendations also rarely imply the need for more resources and oversight reduction, in line with typical member states’ interests in optimizing IO resources and maintaining control over the IO management. Conversely, in organizations where IO management holds sway over evaluation units' resources, the recommendations take on a broader and more ambiguous tone. These recommendations are more likely to advocate for increased resources for the organization and reduced oversight.
Moreover, pressure from stakeholders can lead to negativity or positivity biases in evaluation findings. We delved into this issue in a separate study, analyzing 1082 UN evaluation reports. The results were striking. Reports commissioned by operative units, such as country offices, who benefit from showing a positive impact of their work, are systematically more positive in their results than reports managed by central evaluation units. These patterns underscore the essence of the politics of evaluation—an ongoing struggle between the independence of evaluators and the exertion of power by stakeholders. The delicate balance between impartial evaluation and the vested interests of member states and organizational management remains a defining challenge.
Institutional design can mitigate political influence
Our research underscores the crucial role of institutional design within evaluation systems. It reveals that evaluation units, despite their purported independence, are influenced by the larger organizational structure. Although the UN seeks to harmonize how evaluation is done across its different agencies, some organizations like the International Monetary Fund, the World Bank, or the UN Development Programme have member states' governance boards controlling evaluation system resources, while others like the International Organization for Migration, the UN Refugee Agency, or UN Women are managed by the administration. In some cases ––such as, UNESCO and the Food and Agriculture Organization––both actors are involved. These differences are significant because stakeholder control over evaluation resources can shape the orientation of evaluators towards either member states or the administration. This, in turn, defines whose interests are more likely to be accommodated when conducting evaluations.
We therefore argue against evaluation systems controlled exclusively by either actor, highlighting the risks of undue management influence or member state pressures.
Towards better evidence-based policymaking
Instead of abandoning or isolating evaluation from stakeholders, the solution to evaluation politics lies in recognizing the influence of surrounding political interests and designing evaluation systems accordingly.
First, we suggest moving towards evaluation systems with mixed control, where both member states and the administration are involved in consulting and approving evaluation staff, agenda, and budget. By consulting and approving these mechanisms collaboratively, such systems create a more equitable and trustworthy environment. This approach safeguards against evaluation becoming a tool solely for the use of one powerful stakeholder and fosters objectivity while enabling a comprehensive assessment of the organization's performance.
Second, there is a need for clearer guidance on the formulation and status of evaluation recommendations. To achieve this, a separate guideline or manual outlining a step-by-step process for crafting recommendations can bolster objectivity and shield recommendations from undue influence.
Finally, we should not seek to build a system-wide evaluation mechanism for the whole UN. Such an approach would struggle to consider the unique characteristics of each program, fund, or agency and maintain stakeholder involvement. At the same time, moving towards a completely decentralized evaluation system, where each operative unit is responsible for its own (self-)evaluation, carries the risk of exacerbating evaluation politics. Instead, it is best to put evaluation into the hands of independent and stand-alone central evaluation units within each organization. This setup fosters more effective and contextually relevant evaluation practices while preserving necessary independence, hopefully contributing to the achievement of the SDGs.
In conclusion, the intricate relationship between evidence and power within the United Nations' evaluation processes underscores the necessity of acknowledging and addressing the political dimensions inherent in such assessments. As highlighted in our research, evaluation reports are not immune to biases and manipulation, often reflecting the interests of influential stakeholders. However, by advocating for institutional designs that balance control between member states and IO management, alongside clearer guidelines for recommendation formulation, we can foster more impartial and effective evaluation systems. Embracing these strategies will not only enhance the credibility and utility of evaluation reports but also bolster evidence-based policymaking, ultimately advancing the UN's mission towards achieving the Sustainable Development Goals by 2030.
About the authors
Vytautas Jankauskas is a Postdoctoral Research Fellow at Zeppelin University, and an Associate Research Fellow at Ludwig Maximilian University of Munich.
Steffen Eckhard is Professor of Public Administration and Public Policy at Zeppelin University. He is also affiliated with the 'Politics of Inequality' Center of Excellence at the University of Konstanz and is a Fellow at the Global Public Policy Institute (GPPi) in Berlin.
This think-piece is based on the authors’ newly published book The Politics of Evaluation in International Organizations (Oxford University Press, 2023).
The views and opinions expressed in this think-piece are those of the authors and do not necessarily reflect the official policy or position of SIPA or Columbia University.