How GIZ uses Results-Based Management in Monitoring and Evaluation

Greg Armstrong

With $16 billion in 1,700 ongoing technical cooperation projects, and more than 19,000 staff working in 130 countries, GIZ is one of the world’s biggest technical cooperation implementing agencies. This article reviews the GIZ role in German international development assistance and, GIZ policies on Results-Based Management, Monitoring and Evaluation.

Who This is For:  Project Managers, Bid Managers
Level of Difficulty:  Moderate to complex
Most useful:  Guidelines on designing and using a results-based monitoring system (RBM system)

The size and scope of German International Assistance

According to Donor Tracker in 2017 Germany was the second largest donor for international development assistance in gross amounts disbursed.

Even given the fact that the $ 24.7 billion budget for international assistance included roughly $6 billion for refugee related expenditures, this still makes Germany in gross terms the biggest donor in Europe, and the second biggest in the world.  It ranked 6th in Europe in terms of % of GNI contributed.

Germany is also the largest contributor to the European Development Fund, the largest component of EU-administered development assistance.  

 The two largest components of this development assistance are managed by  KfW Development Bank and its subsidiary DEG, the German Investment Corporation and BMZ – the Ministry of Economic Cooperation and Development.

The Role and Function of BMZ 

While much of this German ODA was managed by KfW Development Bank, and other government Ministries,  BMZ - The Federal Ministry of Economic Cooperation and Development, administered  the largest portion of the budget -roughly 37% of the ODA budget in 2017, and that was predicted to rise to 49% ( $10.7 billion) in 2018.

2018 Budget Allocations - BMZ
Click to enlarge

The largest component of the BMZ aid budget in 2018 was allocated to bilateral development cooperation.

Of the 8.5 billion Euro ($US 9.5 billion administered by BMZ, in 2017 several billion was provided for
financial cooperation, some through the European Union’s aid mechanisms, some to the World
Bank, and the regional development banks, to foundations and civil society organizations.  Some was
also provided to a wide range of United Nations agencies. 

Germany is, for example, the largest government contributor to the UNDP.

Comparison of government contributions to UNDP 2017
(Click to enlarge)

And Germany supported close to 2,000 UNDP projects through UNDP regular resources and another 113 UNDP projects directly in 2017.  

UNDP projects supported by Germany in 2017
(Click to enlarge)

GIZ, as an implementing agency itself received roughly 2.6 billion Euros  (close to $3 billion) in 2017, roughly 2.5 billion coming from BMZ and other German ministries, the rest from organizations such as the European Union, U.N. agencies, foundations or private sector companies, for the implementation of technical cooperation activities and between 2015-2017 was the largest recipient of Europeaid contracts, and although no longer the single largest recipient in  January 2019 remained  in the top 3.

List of top EuropeAid contractors 2015-2017

Over the past 5 years  GIZ has been awarded 218 EuropeAid contracts worth roughly $450 million.

5-Year total of GIZ contracts with EuropeAid - as of January 2019

The Role of GIZ as an Implementing Agency

Aid Agency Results-Based Management Policies: Switzerland, New Zealand,The World Bank, the Netherlands,the United Kingdom, Canada and Sweden

Greg Armstrong

[Updated July 2019]


The OECD has produced 7 case studies on how Results-Based Management policies are used by the World Bank, and the international aid agencies for Switzerland, Sweden, Canada, the United Kingdom,  The Netherlands and New Zealand.  These guides will not help anyone manage a project, but they do provide a useful comparative overview of the intentions of these agencies as they seek to create usable, comprehensible Results-Based Management frameworks.

OECD Results-Based Management Case Studies

Level of Difficulty:  Moderate-Complex
Length: 10-15 pages for each case study, 33 pages for the synthesis
Primarily useful for: Implementing agency managers
Most useful: Annex 1 of the synthesis document
Limitations:  The studies present one side of the analysis – from the agency management, not from users


In 2016 and 2017 OECD produced 6 very short case studies (all PDF) on how Results-Based Management is used for agency management in planning and reporting on aid projects in the World Bank, the Swiss Agency for Development Cooperation, the New Zealand Aid Programme,  the Swedish International Development Agency, Global Affairs Canada,  the United Kingdoms’ Department for International Development, and The Netherlands Ministry of Foreign Affairs.  OECD also produced a useful synthesis report.

Utility and Limitations of the OECD country case studies

For any implementing agency manager  considering either bidding for a project funded by one of these agencies, or for anyone working on a multi-donor project, trying to understand what drives different agencies’ results agenda, each of these guides can provide some superficial introductory summaries of what the insiders in the agencies have to work with, and what pressures may be on them as they manage their agencies programmes. 

But aid agency administrators often think their own agency’s approach to RBM is easier to understand than do the implementing partners, who have to translate the often arcane Results-Based Management policies into practical plans and reports.

These are very short case studies, based on documentary analysis and interviews with aid agency managers, and therefore they present only one side of the story on each agency’s approach to results-based management.

Those who want detailed guidance on how to implement the Results Based Management frameworks in any practical way will not find it in these reports.  That guidance is, for some, but not all of the agencies, available from their websites.

Links to Aid Agency Results-Based Management Guides and Handbooks

 [Update:  A few of the studies, such as those on the Netherlands, The United Kingdom and Sweden, still provide more information than anything  publicly available. But, since these case studies were originally produced, and since I originally wrote this review in March 2018, a number of the links to aid agency sources have expired, or disappeared.  Those who want more detailed guidance on how the different agencies incorporate Results-Based Management in their work, may in some cases find them (as of June 2018) at these links:

• How the World Bank uses results scorecards in Results-Based Management 
• How The Swiss Agency for Development Cooperation measures results 
• How the New Zealand Aid Programme Manages for Results  
• Results-Based Management at the Swedish International Development Agency  
• The most recent Global Affairs Canada Results-Based Management Guide, tools and checklists 
• The somewhat dated 2014 Department for International Development results framework from the United Kingdom 
• The basically (in English at any rate) opaque  Dutch Development Results site unfortunately is less useful than the OECD case study.]

Lessons Learned from the Case Studies

Of potentially more interest from a comparative perspective is the 31- page synthesis report Strengthening the Results Chain by Rosie Zwart.  

OECD Synthesis Report on Results-Based Management

This document summarizes and analyses the challenges facing the different agencies, in terms of how they link their internal results frameworks to long term international development results, how they use standard indicators, and the problems associated with this, how the results and indicators contribute to accountability, how attribution of credit for results is handled in each agency, how they use narratives to make sense of the results frameworks, and the extent to which results reporting contributes to  any meaningful learning – and change, within the agencies

Short summaries of how aid agencies use RBM
[click to enlarge]

Readers may find the Annex to the report, which summarizes many of these issues by aid agency, useful, before deciding whether to read the individual country reports.

RBM Comparison Chart for 7 Aid Agencies
[Click to enlarge]

The bottom line:  These case studies can provide very useful superficial overviews of the challenges facing different aid agencies as they implement Results-Based Management, but more useful guidance on each is often available directly from the agency websites for those with the time, and the motivation, to use them.

_______________________________________________________

GREG ARMSTRONG

Greg Armstrong is a Results-Based Management specialist who focuses on the use of clear language in RBM training, and in the creation of usable planning, monitoring and reporting frameworks.  For links to more Results-Based Management Handbooks and Guides, go to the RBM Training website. 

Evaluability Assessments and Results-Based Management: 8 Guides

by Greg Armstrong

Evaluability Assessments answer the question:  Is there enough available information to justify the time and cost of a full-scale evaluation? If they are done early enough during implementation, they can identify basic problems in design which can guide remedial action.

One old study set the stage for 7 recent detailed Guides which synthesize earlier work and provide useful advice on whether, when and how to undertake an evaluability assessment – or to use other approaches to assessing project or programme design integrity.

Evaluability Assessment Guides

Level of difficulty: Moderate
Length: 3-72 pages
Primarily useful for: Evaluation managers, RBM specialists doing evaluation assessments
Most useful: DFID Guide to Planning Evaluability Assessments
Most interesting:  Program Management and the Federal Evaluator (1974)
Limitations:  These guides tell us what needs to be done, but they require people with the process skills to do it all.

The History of Evaluability Assessments

Evaluability assessments, pre-assessments or exploratory evaluations, as they were known in some cases 40 years ago, have been used for many years in public health research where the term “evaluable” refers to ‘Patients whose response to a treatment can be measured because enough information has been collected”,  education,  and justice  (PDF) and the term has academic antecedents in testing mathematical propositions, far back into the 19th Century. 

The Foundation Evaluability Assessment Document

The foundation evaluability concepts for social programmes were initially presented, from what I can see in a 1974 Urban Institute study, one of many emerging during implementation studies of the Johnson administration's Great Society Program,  published as "Program Management and the Federal Evaluator" in the Public Administration Review, included in a 1977 volume of Readings in Evaluation Research, available through Google Books. 

The Foundation Evaluability Assessment article

It advocated looking at issues which are at the heart of current guides to evaluability assessment - and for that matter, at the heart of solid results-based design:

• Whether there is a clearly defined problem addressed in design
• If the intervention is clearly defined
• If the short and longer term results are defined clearly enough to be measurable
• if “the logic of assumptions inking expenditure of resources, the implementation of a program intervention , the immediate outcome to be caused by that intervention, and the resulting impact” are “specified or understood clearly enough to permit testing them”.
• Whether managers are capable of and motivated to using performance data for concrete management decisions.

While there are many other guides to evaluability assessment which are much more detailed of more practical utility today, the most important core concepts are here in this 1974 study, and this short, 43-year old article is still worth reading for its insights on clarity on language, assumptions and results.  

Leonard Rudman’s later 1980 book Planning Useful Evaluations - Evaluability Assessment  set out a detailed approach to dealing with all of these issues.

Changes in Evaluability Assessment Utilization

Evaluability Assessments started out as ways of improving the design of evaluations, to increase the chances they will be useful to the people who are funding and implementing activities, and in recent years, with a substantial increase in the number of evaluations of international development projects, many agencies, including UNICEF, the World Bank, UNDP and numerous bilateral donors have incorporated evaluability assessments as part of the project cycle, after design, during implementation and before a decision is made to pay for a full-scale evaluation.  

Many Evaluability Assessment Guides have been produced in recent years, but here are just a few worth noting.