google-site-verification: googlefccb558c724d02c7.html

Translate

Wednesday, January 09, 2019

How GIZ uses Results-Based Management in Monitoring and Evaluation

Greg Armstrong

With $16 billion in 1,700 ongoing technical cooperation projects, and more than 19,000 staff working in 130 countries, GIZ is one of the world’s biggest technical cooperation implementing agencies. This article reviews the GIZ role in German international development assistance and, GIZ policies on Results-Based Management, Monitoring and Evaluation.
Map showing in which countries GIZ is active










Who This is For:  Project Managers, Bid Managers
Level of Difficulty:  Moderate to complex
Most useful:  Guidelines on designing and using a results-based monitoring system (RBM system)



The size and scope of German International Assistance


According to Donor Tracker in 2017 Germany was the second largest donor for international development assistance in gross amounts disbursed.


Even given the fact that the $ 24.7 billion budget for international assistance included roughly $6 billion for refugee related expenditures, this still makes Germany in gross terms the biggest donor in Europe, and the second biggest in the world.  It ranked 6th in Europe in terms of % of GNI contributed.

Germany is also the largest contributor to the European Development Fund, the largest component of EU-administered development assistance.  

 The two largest components of this development assistance are managed by  KfW Development Bank and its subsidiary DEG, the German Investment Corporation and BMZ – the Ministry of Economic Cooperation and Development.

The Role and Function of BMZ 


While much of this German ODA was managed by KfW Development Bank, and other government Ministries,  BMZ - The Federal Ministry of Economic Cooperation and Development, administered  the largest portion of the budget -roughly 37% of the ODA budget in 2017, and that was predicted to rise to 49% ( $10.7 billion) in 2018.

2018 Budget Allocations - BMZ
Click to enlarge
The largest component of the BMZ aid budget in 2018 was allocated to bilateral development cooperation.

Of the 8.5 billion Euro ($US 9.5 billion administered by BMZ, in 2017 several billion was provided for
financial cooperation, some through the European Union’s aid mechanisms, some to the World
Bank, and the regional development banks, to foundations and civil society organizations.  Some was
also provided to a wide range of United Nations agencies. 

Germany is, for example, the largest government contributor to the UNDP.



And Germany supported close to 2,000 UNDP projects through UNDP regular resources and another 113 UNDP projects directly in 2017.  
Map showing the location of 2,087 UNDP projects supported by Germany in 2017
UNDP projects supported by Germany in 2017
(Click to enlarge)


GIZ, as an implementing agency itself received roughly 2.6 billion Euros  (close to $3 billion) in 2017, roughly 2.5 billion coming from BMZ and other German ministries, the rest from organizations such as the European Union, U.N. agencies, foundations or private sector companies, for the implementation of technical cooperation activities and between 2015-2017 was the largest recipient of Europeaid contracts, and although no longer the single largest recipient in  January 2019 remained  in the top 3.
Chart showing the top 10 EuropeAid contractors between 2015-2017
List of top EuropeAid contractors 2015-2017
Over the past 5 years  GIZ has been awarded 218 EuropeAid contracts worth roughly $450 million.

5-Year total of GIZ contracts with EuropeAid - as of January 2019

The Role of GIZ as an Implementing Agency


Monday, March 12, 2018

Aid Agency Results-Based Management Policies: Switzerland, New Zealand,The World Bank, the Netherlands,the United Kingdom, Canada and Sweden

Greg Armstrong

[Updated July 2019]


The OECD has produced 7 case studies on how Results-Based Management policies are used by the World Bank, and the international aid agencies for Switzerland, Sweden, Canada, the United Kingdom,  The Netherlands and New Zealand.  These guides will not help anyone manage a project, but they do provide a useful comparative overview of the intentions of these agencies as they seek to create usable, comprehensible Results-Based Management frameworks.

Case Studies on how 6 aid agencies use Results-Based Management
OECD Results-Based Management Case Studies


Level of Difficulty:  Moderate-Complex
Length: 10-15 pages for each case study, 33 pages for the synthesis
Primarily useful for: Implementing agency managers
Most useful: Annex 1 of the synthesis document
Limitations:  The studies present one side of the analysis – from the agency management, not from users


In 2016 and 2017 OECD produced 6 very short case studies (all PDF) on how Results-Based Management is used for agency management in planning and reporting on aid projects in the World Bank, the Swiss Agency for Development Cooperation, the New Zealand Aid Programme,  the Swedish International Development Agency, Global Affairs Canada,  the United Kingdoms’ Department for International Development, and The Netherlands Ministry of Foreign Affairs.  OECD also produced a useful synthesis report.


Utility and Limitations of the OECD country case studies


For any implementing agency manager  considering either bidding for a project funded by one of these agencies, or for anyone working on a multi-donor project, trying to understand what drives different agencies’ results agenda, each of these guides can provide some superficial introductory summaries of what the insiders in the agencies have to work with, and what pressures may be on them as they manage their agencies programmes. 

But aid agency administrators often think their own agency’s approach to RBM is easier to understand than do the implementing partners, who have to translate the often arcane Results-Based Management policies into practical plans and reports.

These are very short case studies, based on documentary analysis and interviews with aid agency managers, and therefore they present only one side of the story on each agency’s approach to results-based management.

Those who want detailed guidance on how to implement the Results Based Management frameworks in any practical way will not find it in these reports.  That guidance is, for some, but not all of the agencies, available from their websites.

Links to Aid Agency Results-Based Management Guides and Handbooks

 [Update:  A few of the studies, such as those on the Netherlands, The United Kingdom and Sweden, still provide more information than anything  publicly available. But, since these case studies were originally produced, and since I originally wrote this review in March 2018, a number of the links to aid agency sources have expired, or disappeared.  Those who want more detailed guidance on how the different agencies incorporate Results-Based Management in their work, may in some cases find them (as of June 2018) at these links:



Lessons Learned from the Case Studies


Of potentially more interest from a comparative perspective is the 31- page synthesis report Strengthening the Results Chain by Rosie Zwart.  

OECD Synthesis Report on Results-Based Management
OECD Synthesis Report on Results-Based Management


This document summarizes and analyses the challenges facing the different agencies, in terms of how they link their internal results frameworks to long term international development results, how they use standard indicators, and the problems associated with this, how the results and indicators contribute to accountability, how attribution of credit for results is handled in each agency, how they use narratives to make sense of the results frameworks, and the extent to which results reporting contributes to  any meaningful learning – and change, within the agencies

Short summary paragraphs on how 7 aid agencies use Results-Based Management
Short summaries of how aid agencies use RBM
[click to enlarge]

Readers may find the Annex to the report, which summarizes many of these issues by aid agency, useful, before deciding whether to read the individual country reports.
A chart comparing how 7 aid agencies use RBM, using 6 criteria
RBM Comparison Chart for 7 Aid Agencies
[Click to enlarge]


The bottom line:  These case studies can provide very useful superficial overviews of the challenges facing different aid agencies as they implement Results-Based Management, but more useful guidance on each is often available directly from the agency websites for those with the time, and the motivation, to use them.

_______________________________________________________











GREG ARMSTRONG
Greg Armstrong is a Results-Based Management specialist who focuses on the use of clear language in RBM training, and in the creation of usable planning, monitoring and reporting frameworks.  For links to more Results-Based Management Handbooks and Guides, go to the RBM Training website

Tuesday, August 15, 2017

Evaluability Assessments and Results-Based Management: 8 Guides

by Greg Armstrong

Evaluability Assessments answer the question:  Is there enough available information to justify the time and cost of a full-scale evaluation? If they are done early enough during implementation, they can identify basic problems in design which can guide remedial action.

One old study set the stage for 7 recent detailed Guides which synthesize earlier work and provide useful advice on whether, when and how to undertake an evaluability assessment – or to use other approaches to assessing project or programme design integrity.

Evaluability Assessment Guides
Evaluability Assessment Guides



Level of difficulty: Moderate
Length: 3-72 pages
Primarily useful for: Evaluation managers, RBM specialists doing evaluation assessments
Most useful: DFID Guide to Planning Evaluability Assessments
Most interesting:  Program Management and the Federal Evaluator (1974)
Limitations:  These guides tell us what needs to be done, but they require people with the process skills to do it all.


The History of Evaluability Assessments


Evaluability assessments, pre-assessments or exploratory evaluations, as they were known in some cases 40 years ago, have been used for many years in public health research where the term “evaluable” refers to ‘Patients whose response to a treatment can be measured because enough information has been collected”,  education,  and justice  (PDF) and the term has academic antecedents in testing mathematical propositions, far back into the 19th Century. 

The Foundation Evaluability Assessment Document


The foundation evaluability concepts for social programmes were initially presented, from what I can see in a 1974 Urban Institute study, one of many emerging during implementation studies of the Johnson administration's Great Society Program,  published as "Program Management and the Federal Evaluator" in the Public Administration Review, included in a 1977 volume of Readings in Evaluation Research, available through Google Books. 
Title page of "Program Management and the Federal Evaluator"
The Foundation Evaluability Assessment article
It advocated looking at issues which are at the heart of current guides to evaluability assessment - and for that matter, at the heart of solid results-based design:
  • Whether there is a clearly defined problem addressed in design
  • If the intervention is clearly defined
  • If the short and longer term results are defined clearly enough to be measurable
  • if “the logic of assumptions inking expenditure of resources, the implementation of a program intervention , the immediate outcome to be caused by that intervention, and the resulting impact” are “specified or understood clearly enough to permit testing them”.
  • Whether managers are capable of and motivated to using performance data for concrete management decisions.
While there are many other guides to evaluability assessment which are much more detailed of more practical utility today, the most important core concepts are here in this 1974 study, and this short, 43-year old article is still worth reading for its insights on clarity on language, assumptions and results.  
Leonard Rudman’s later 1980 book Planning Useful Evaluations - Evaluability Assessment  set out a detailed approach to dealing with all of these issues.

Changes in Evaluability Assessment Utilization


Evaluability Assessments started out as ways of improving the design of evaluations, to increase the chances they will be useful to the people who are funding and implementing activities, and in recent years, with a substantial increase in the number of evaluations of international development projects, many agencies, including UNICEF, the World Bank, UNDP and numerous bilateral donors have incorporated evaluability assessments as part of the project cycle, after design, during implementation and before a decision is made to pay for a full-scale evaluation.  

Many Evaluability Assessment Guides have been produced in recent years, but here are just a few worth noting.

RBM Training

RBM Training
Results-Based Management

Share on LinkedIn

Subscribe to this blog by email

Enter your email address:

Delivered by FeedBurner

 
Read the latest posts