Global Affairs Canada Results-Based Management Guides: 1998-2021

 

 Greg Armstrong - Last updated: April 2 2023

The Global Affairs Canada suite of Results-Based Management guides, tools and checklists are among the most practical guides available for the design, management, monitoring and results reporting for projects funded by any international agency.  And, of course, they are essential for anyone bidding on or implementing projects funded by Global Affairs Canada.   

As strong as the project-level RBM guides are, however, the agency-wide results framework for GAC is surprisingly weak.

This article reviews the most recent changes made to the GAC RBM system between 2016-2023, considers how flexible the GAC Results-Based Management system is, in practice, and looks at the state of agency-wide Global Affairs Canada results reporting,  in early 2023.

GAC RBM Guides: 1998-2021

Contents

• GAC RBM Guides
• GAC Results Terminology
• The Global Affairs Canada Results Chain
• Key GAC RBM Tools:
• The GAC Logic Model
• The GAC Performance Measurement Framework
• The GAC Risk Framework
• RBM Templates
• The Output-Activities Matrix
• A New GAC Results Reporting Format
• Limitations of the GAC RBM Guides
• Is the GAC RBM system adaptive?
• A Weak Agency-Wide GAC Results Framework
• Changing GAC Indicators

GAC RBM Guides

1996-2008

CIDA RBM Guides 1998-2008 (click to enlarge)

The Canadian aid agency – formerly known as CIDA – now part of Global Affairs Canada (GAC as it is known in the agency) – adopted Results-Based Management in 1996.  While there were several  academic and internal early CIDA commentaries on RBM, the first more or less complete CIDA RBM guide still available, published in 2000, was  the easy to read  RBM Handbook on Developing Results Chains:  The Basics of RBM as Applied to 100 Project Examples (PDF).

This was followed in 2001 by a useful and user-friendly 97-page guide to using RBM in writing a project implementation plan  produced for CIDA by Peter Bracegirdle. I reviewed  that RBM guide several years ago, and it is still available here and several other sites. In 2002, a short reporting guide was produced for CIDA Asia Branch by Universalia and the Conference Board of Canada.

Despite changes to CIDA results terminology in 2008, and the posting  in 2009 of issue-by-issue guides on the CIDA, later DFATD and Global Affairs Websites,  under the general title Results-Based Management Tools at CIDA: A How-to Guide,  the resources available to people trying to use Results-Based Management in both the design and implementation of Canadian projects, were limited. 

Trainers had to paste together different documents  on logic models, indicators, risk and indicators obtained off the website, to produce a coherent, if somewhat jargon-laden ad hoc RBM guide of roughly 45 pages.

Many people instead continued to use the 2001 PIP Guide  [A Results Approach to the Implementation Plan] as the most effective of the CIDA/DFAT/GAC guides up until 2016, not just for developing implementation plans at project inception, but, making adaptations for terminology changes, as an aid to annual work planning,

Back to Table of Contents

2016

GAC 2016 RBM Guide

In 2016 a new group within Global Affairs Canada, called the Results-Based Management Centre of Excellence, produced a comprehensive and very practical new  105- page Results-Based Management guide, under the title Results-Based Management for International Assistance Programming : A How-to Guide (PDF).

The 2016 Guide is also available in French as La gestion axée sur les résultats appliquée aux programmes d’aide internationale: Un guide pratique .

This most recent RBM guide has a monochrome, somewhat boring appearance, and while  it includes a lot of material from earlier materials used since 2008, it also has a substantial number of new clarifications. The new material makes the 2016 RBM guide a much more practical RBM tool than previous versions published since 2008.

2017-2023

GAC produced a minor update of the original 2018 RBM guide in June 2022 and it first appeared online in September 2022. That update changed very little of substance from the original guide.  The 96 changes I found were for the most part additions of terms directed to GAC staff, emphasizing that the RBM tools should be applied not just to individual projects, but to the portfolio of projects they manage. There is now also an additional graphic in the section of the guide dealing with implementation (below) which may clarify for some readers what the text explains.  There has also been a change in the colour scheme used for the GAC results chain (see below).

The only substantive change I found was the deletion of a ten-page glossary of 50 RBM terms used by GAC at the end of the original guide (p. 92-103).  That is unfortunate, because I think having those terms together in one place provides useful information for people who might get lost in the inevitable jargon which accompanies management guides and for that reason I suggest that people who are not working inside GAC itself may find the original 2016 RBM guide more useful.  

The GAC RBM page also contains a 2018 Results Reporting Guide for Partners (PDF), which expands on the reporting component of the main How-to guide.  The GAC results reporting guide recognizes that circumstances change, and with this, the need for some redesign may occur during the project life cycle.  

This reporting guide, and a number of RBM checklists and tip sheets on developing, assessing or reviewing theories of change, logic models, indicators in general, a performance measurement framework, gender equality results and indicators, how RBM can be applied in human rights projects and other topics, have moved the RBM process in Global Affairs Canada to a slightly more iterative and incremental rather than a rigidly prescriptive framework.  

Some of these tip sheets and checklist are used by GAC staff as they assess proposals, and monitor the implementation of projects, so they are worth looking at for anyone seeking GAC funding or implementing a GAC-funded project.

GAC RBM Checklists and Tip sheets

The question of whether the Canadian aid agency's RBM framework is too rigid, or adaptive enough to suit changing circumstances, is something I will deal with in more detail later in this article, below. 

CIDA was absorbed within the Canadian Department of Foreign Affairs (now GAC) in 2013. Yet, by 2021, one prominent newspaper columnist writing in the Globe and Mail noted that "The department has never been able to integrate aid and trade with diplomacy." 

As the independence of the Canadian aid programme disappeared through successive Conservative and Liberal governments, the number of professionals on staff with experience in designing development projects decreased, as they left the agency or moved into trade or diplomatic sections of the Ministry.   

The process of designing projects became incrementally more adaptive and iterative, particularly in its partnerships with civil society organizations.  

By 2019, Canadian aid channelled through domestic and international Civil Society organizations totalled roughly C$1.4 billion.   Much of this has been allocated through competitive calls for proposals, or by CSO's submitting unsolicited proposals.

The burden for design has gradually shifted in some, but not all programmes, to the potential grant recipients or implementing agencies, and they have in often been expected to conform to the design and M&E standards in the CIDA/GAC RBM guides.  

This meant figuring the RBM processes out for themselves, using the guides, finding technical help sometimes from the people in the aid section of the Foreign Affairs Department agency who might previously have worked on project design, or from consultants - like me.

One example of this burden came with the development of new guidelines and requirements for unsolicited proposals from civil society organizations in 2019.  A preliminary proposal form for unsolicited proposals used by GAC in 2019 and early 2020, was quite demanding of applicants. 

Incorporating Theory of Change in the 2019 GAC Preliminary Proposal Form  (click twice to enlarge)

The requirements for developing the Theory of Change were strict, complicated by the fact that applicants could use only text, not a diagram to assist in the explanation of problems, causes, risks and results, although a Logic Model was permitted - and required.  

And, while formal consultations with stakeholders were not strictly required for this preliminary stage, this theory of change had to be developed in a participatory manner with counterparts in the field, before a full proposal was developed, perhaps eventually approved, and then funds provided for activities.  

The preliminary proposal form was usually between 12-15 pages in length -  and it was just the first step.  If the preliminary proposal appeared targeted to GAC's priorities and well structured, applicants were invited to submit a full proposal, with more detail.

The COVID-19 pandemic complicated this, or course, and in July 2020, GAC reintroduced a much simpler and less demanding initial procedure - formalized in September 2020  The new procedure substitutes a 5- page concept note for the more complicated preliminary proposals.  This is still part of a two-stage process requiring for those getting past the initial review, a full proposal.  

In theory, although perhaps not always in practice, a full theory of change still has to be produced before funding is approved. It remains to be seen at this stage whether the new approach produces projects with a coherent theory of change.

All of these reporting guidelines, checklists, application forms and tip sheets, are however, based on the primary 2016 GAC RBM Guide.  It is visually bland, monochromatic and unappealing, but the contents are detailed and, with the addition of the new reporting guide and the RBM tip sheets and checklists, it is an important, practical resource for project design, management and monitoring.

Table of Contents

Who the GAC RBM Guides are for:

These guides will be of use beyond the primary intended audience which was originally implementing agencies, staff of Global Affairs and those working with them on project and project design.  

While some of the background information describing the relationship of the RBM How-To guide to other Canadian government policies will be of little use or interest to anyone outside of the Canadian government, there is a lot of material here which could help implementing agencies and partners working on Canadian – funded projects, to work more effectively.  

And it is easy to see, with the practical step-by-step focus on problem identification, theory of change, risk management, and particularly moving from a Logic Model to specific project design components, how these guides will be useful to people designing projects for any bilateral or multilateral agency, regardless of the funding source.  

Back to Table of Contents

The GAC Results Chain in 2016

GAC Results Chain in 2022

GAC Results Terminology

The Global Affairs approach to Results-Based Management has been, since 2008, an improvement over that of several other agencies, and on its own earlier terminology.  It now limits the labelling of results  to the term “outcomes”.  

CIDA, in an earlier phase, used the terms "Outputs", "Outcomes" and "Impact" to describe different levels of results.  While the term “objectives” is in peripheral evidence in the 2016 RBM guide  it never appears in the functional tools such as the GAC Results Chain, the Logic Model, or the Performance Measurement Framework.  

There is no confusion here with results being described as objectives, purposes, impacts or goals, terms which, for some agencies, are used almost interchangeably at different stages of a results chain, along with Outputs, Outcomes and Results.  

Such vague language is something that often leads to genuine confusion as implementing agencies, partners and beneficiaries try to describe results, and distinguish them from activities.  

For Global Affairs Canada, as for CIDA before it,  all results are changes:  Not completed activities as some U.N. agencies confusingly label low-level results – but changes in the short term in capacity, in understanding, skills, or access to services.  

At higher levels results are seen as changes in the behaviour, practice and performance, of change agents or of people who are the long-term beneficiaries.  All of these changes are, in theory, designed to contribute to even longer-term changes in important life issues such as income, food security, health, security, status of women, levels of suffering or human rights.

The Global Affairs Canada Results Chain

The GAC results chain, in English and French, for Canadian aid projects has, since 2008, looked like this:

The GAC results chain in English and French 
(click to enlarge)

It is interesting to note that in the French language version of the Global Affairs Canada RBM guide, what are called, in English, "Immediate Outcomes", "Intermediate Outcomes" and "Ultimate Outcomes" in are, in French, just immediate, intermediate or ultimate “results”.

Comparing terms for GAC results in French and English

The differences between the English and French reflect the Treasury Board of Canada Results-Based Management Lexicon  which standardizes the results frameworks for Canadian government agencies.

I do not find the addition in English of the term "Outcomes" - instead of just labelling them results, to be helpful.

As someone who works regularly to help people understand RBM in other languages, defining a result as a “change” is something that can be easily translated into any language, not just for government officials or field workers, but for villagers and other beneficiaries.  

 "Outputs" and "Outcomes" are both words used in English in many different forms, which causes problems of understanding even for native English speakers working on RBM, including staff in donor agencies. 

In some other languages, while “change” is always understood, special terms have to be devised to describe Outputs or Outcomes and the differences between them.  

As I have argued elsewhere, clear language is always preferable, if we want people to actually use Results-Based Management (or any other organizational innovation)  in practice. 

I doubt, given the organizational context, and GAC's place in the Treasury Board scheme of things, that there is anything GAC RBM specialists can do about this, however.

Outputs - not Results

This version of the RBM guide provides improved operational clarity in the definitions of what are not results – inputs, activities, and particularly the products of activities – clearly labelled as “Outputs”.

Outputs are described as “Direct products or services stemming from the activities of an organization, policy, program or project.”    

Examples of GAC Outputs  (click to enlarge)

Those who have examined or worked with the Results terminology used by U.N. agencies will note  difference between this, and the common definition of Outputs  used until recently by many U.N. agencies [my emphasis added]:

"Specific goods and services produced by the programme. Outputs can also represent changes in skills or abilities or capacities of individuals or institutions, resulting from the completion of activities within a development intervention within the control of the organization. " [Results-Based Management in the United Nations Development System, 2016, p. iii] 

 "Outputs are changes in skills or the abilities and capacities of individuals or institutions, or the availability of new products and services that result from the completion of a development intervention." [United Nations Development Assistance Framework Guidance, Feb 2017, p. 24] 

In practical terms the confusion caused by mixing products and actual changes in capacity into one common category, has meant that only the most serious U.N. agency managers have actually reported on the more difficult changes in capacity. 

Their less….ambitious… colleagues have satisfied themselves, although not their bilateral partners, by reporting on completed activities – numbers of people trained, handbooks produced, schools built - as real results. 

This has proven to be a real source of frustration to bilateral donors contributing to U.N. agency activities. 

Many of these bilateral  British, Australian, German, Canadian aid agencies and others, need to report on changes, such as increased skills, better performance, increased learning by students, or improved health, security or income - and not just on activities completed.

Results Level hierarchy

GAC results - three forms of Outcomes - are organized in a Logic Model, representing different types of changes.

Immediate Outcomes (or Résultat immédiat)

Types of changes in GAC Immediate Outcomes

Immediate Outcomes are, for Global Affairs Canada: 

“A change that is expected to occur once one or more outputs have been provided or delivered by the implementer. In terms of time frame and level, these are short-term outcomes, and are usually changes in capacity, such as an increase in knowledge, awareness, skills or abilities, or access.. to... [services] ...among intermediaries and/or beneficiaries.  ...Changes in access can fall at either the immediate or the intermediate outcome level, depending on the context of the project and its theory of change. " [GAC RBM How-to Guide, p. 72]

Intermediate Outcomes (Résultat intermédiaire) 

GAC defines Intermediate Outcomes as 

"A change that is expected to logically occur once one or more immediate outcomes have been achieved. In terms of time frame and level, these are medium-term outcomes that are usually achieved by the end of a project/program, and are usually changes in behaviour, practice or performance among intermediaries and/or beneficiaries."  [p. 71]

Ultimate Outcomes (Résultat ultime) 

Types of changes in GAC Ultimate Outcomes  (click to enlarge)

GAC Ultimate Outcomes are defined as 

"The highest-level change to which an organization, policy, program, or project contributes through the achievement of one or more intermediate outcomes. The ultimate outcome usually represents the raison d'être of an organization, policy, program, or project, and it takes the form of a sustainable change of state among beneficiaries." [p. 71]

Among the many useful small changes to the way these definitions work, is the admonition that such long-term changes should not refer to generic changes in the country’s circumstances (such as improved GDP), but should deal with real changes in the lives of real people – in health, learning, security and other areas which can be demonstrated with indicator data.

Back to Table of  Contents

Key GAC RBM Tools: Logic Model, Output-Activities Matrix, Performance Measurement Framework 

CIDA in 2008 moved from the familiar Logical Framework, which combined results, indicators, assumptions and risk in a visually (and often intellectually)  confusing  manner, to disaggregation of the main elements of the Logical Framework into three distinct elements:

The GAC Logic Model  

Based on a theory of change exercise, the Logic Model visually illustrates how different elements are intended to be combined to contribute to short-term, medium-term and long-term changes as this example from  a 2015 Request for Proposals  (p. 55) for an economic development project in Indonesia illustrates

The GAC Performance Measurement Framework

The Performance Measurement Framework presents indicators, targets data collection methods, frequency of data collection and schedules for different levels of results, as the 2013 example, above, illustrates.

The GAC Risk Management Framework

GAC Risk Assessment Scale

The GAC Risk Management Framework is a relatively simple combination of tools including a risk assessment scale, risk register, instructions and definitions for 11 primary operational, financial, development and reputational risks.  

The GAC Risk Framework combination of tools
(click to see details)

When the tools are used by managers,they are not always updated, however, and some project managers find the process of accurately defining risk difficult.

For those project managers interested in a more nuanced and complex approach, the Treasury Board of Canada Project Complexity and Risk Assessment Tool, provides some useful insights into how government departments are expected to assess risk on proposed projects.  This tool is adapted, itself from the Software Engineering Institute's Continuous Risk Management Guidebook

This more detailed risk assessment tool has 64 questions including rating scales for risk in areas such as procurement, project governance, funding, scheduling, geography, socioeconomic factors, human resources requirements, public perception, communications, project complexity, and integration with other activities, among others.  

Not all of these will be relevant to an individual project, but many of these could, and should stimulate some reflection for both those designing projects, and managing them.

Back to Table of Contents

RBM templates

GAC has several templates which, combined could, with some limitations, simplify the mundane if not the intellectual tasks, of distinguishing between and recording the links between Activities, Outputs, and Outcomes in the Logic Model, and in recording agreements on indicators. 

The positive side of these templates is that they standardize what is produced, and make it difficult to inadvertently omit or change the wording of results, as we move from a Logic Model to the development of activities and indicators.  

Outcome & Outcome statements entered into the GAC Logic Model

Project Design - Outcome and Outputs from the Logic Model transferred to the Outputs and Activities Matrix  (click to enlarge)

The negative side of these templates – form-filling PDF files, which restrict reformatting, is that they can be difficult to work with if the forms are being projected onto a screen and being used as the basis for discussion in large Logic Model and indicator workshops, where using the suggested “sticky notes” is not practical. 

In those situations  reformatting is often necessary to accommodate changes as the discussion occurs – as new columns and notes need to be added to remind participants how these changes have evolved, and what needs to be done. In my experience this is not possible with these forms. 

This could be handled subsequent to a workshop in additional text, but it is best to get these things on record quickly, while the discussion is taking place.  In these situations I have found word processing programmes such as Microsoft Word or Google Docs much easier to work with than PDF or spreadsheet formats. 

Another difficulty is that the Logic Model template limits users to 3 Intermediate Outcomes, and in many projects there could be an argument that there will be more, if a theory of change analysis leads to this.  

These templates are convenient, therefore, for GAC's own internal management and standardized aggregation of results across hundreds of projects, but they can be restrictive to individual users.    And as I will discuss later, GAC has not done a very good job of actually aggregating results, in any case.

The templates for these tools are  not part of the actual GAC RBM Guide itself - at least not as of this writing, and although download links are provided either in the text or at the Global Affairs website, it is not easy, if you are using Chrome,  or Microsoft Edge to actually see the templates or download them even if you own Acrobat. 

This can be circumvented in Firefox by clicking on the download button and selecting the option to open the file in Adobe Acrobat or Reader, but in Edge and Chrome opening a readable copy or downloading anything beside an error page is very difficult, not just for me, but for many other people I have talked to.  However, if you want to give it a shot, there are, theoretically links to the templates for the Logic Model, the Performance Measurement Framework, and the Risk Table  - but you may get an error message.  

This may be just as well, because while the tools are useful the PDF format is too restrictive to be used easily. 

Additional tools for organizing data and reporting results are suggested in the GAC results reporting guide, including separate worksheets to help organize information on Outcome and Output indicators, prior to writing a narrative report, and guidelines on how to put information together in a results report.   There are so far no downloadable templates for these, but the formats are easy to copy and use in a standard word processing programme - something I also suggest for the Logic Model, the PMF and risk tools.

Improved operational clarity

All of the basic tools remain essentially the same as they were in 2008,  but the improvement over earlier CIDA guides produced after 2008 is that there is increased clarity in this document, and in the Results Reporting Guide, about how to use the Logic Model, Output-Activities Matrix and Performance Measurement Framework,  in practical terms in project design, implementation, monitoring and results reporting.  

The 2001 PIP Guide remains a useful tool, as it had more detail on some design issues such as the Work Breakdown Structure, activity scheduling, budgeting and stakeholder communication plans.  But the new Guide, working with earlier material after 2008, and with new examples, contains useful new clarifications throughout the document.  

Distinguishing between Outputs and Activities: 

the Output-Activities Matrix  

This sounds mundane, but there has been confusion in some Logic Models about whether Outputs were just completed activities or something more.  So, in that approach, an activity might be “Build wells” and the Output would be “Wells built”.  This is of no use at all in helping project managers mobilize and coordinate the resources and individual activities necessary to really put the wells in the ground.  

I have always found the CIDA (2001) Output-Activity Matrix to be a useful bridge between the theory of the Logic Model, and the need for concrete focus in work planning. This document makes this link between the Logic Model and operational reality, and the link to results-based scheduling, clearer.

The current GAC template for the Logic Model automatically populates the Output-Activities Matrix with Outputs, preparatory to the users figuring out what activities are necessary to achieve them.  

An example of the Outputs-Activities Matrix after Activities are added

This form itself provides the foundation for results-based scheduling of project activities as the next logical step in project design, particularly at the stage of project inception.

Results-Based Scheduling Format

Table of Contents

Examples of how to phrase Outcomes in specific terms (syntax)

The GAC RBM framework has several criteria for developing precise result - Outcome statements, reflecting the fact that these are supposed to represent changes of some kind for specific people, in a specific location.  The RBM guide provides illustrations of two ways this can be done:

Syntax Structure of an Outcome Statement - Global Affairs Canada

The Guide also provides examples of strong and weak Outcome statements, with suggestions on how they can be improved.

Examples of strong and weak Outcome statements

A new GAC Results Reporting Format

The RBM How-to Guide and the GAC results reporting guide provide useful new (for GAC projects)  formats for results reporting.  In the past different projects have reported in a wide variety of ways, often forcing readers to wade through dozens of pages of descriptions of activities, in a vain attempt to find out what the results are.  

This new results reporting format puts results up front, in a table, emphasizing indicator data, with room for explanations in text, below.  

The results reporting guide also includes separate worksheets to help organize information on Outcome and Output indicators, prior to writing a narrative report and guidelines on how to complete such a narrative report.  These are available only as models, not as downloadable templates.

Global Affairs Canada Results Reporting Forms  (click to enlarge)

The main GAC RBM How-To Guide provides other examples of how results should be organized for reporting in table form.

Suggested Results Reporting Format - emphasizing progress against indicators and targets (click to enlarge)

These and other results reporting tips can be found in section 3 – Step by Step Instructions on results-based project planning and design (p. 66-85) and section 4 – Managing for Results during Implementation (p. 86-92) but others are spread throughout the document, and for that reason it is useful to read the whole document, even if users are familiar with past CIDA/GAC documents.  The 2022 revision of the RBM guide contains one additional graphic putting together the steps involved in implementing RBM.

Using indicator data to manage for results - GAC update 2022

Back to Table of Contents

Limitations of the GAC RBM Guides

I see two areas where further improvements could be made, some of which could be done informally, and some which, given procedures in the Government of Canada's Treasury Board, are perhaps beyond the scope of the GAC RBM group’s control.

1. Dealing with the Implications of RBM for operations and funding

I have seen very small civil society organizations face lengthy processes of data collection and report revisions, to comply with donor agency RBM requirements for relatively inexpensive projects. But at the same time donor agencies themselves - and this means most donors - often do not deal realistically with the implications of their own guidelines for project budgets.

The costs of baseline data

Take baseline data collection, for example.  The GAC Guide sections on Indicators and the Performance Measurement Framework (p. 52-64) are generally quite practical, and make the very valid point that baseline data for indicators must be collected before targets can be established, and results reported on.    

I agree completely that this is the most useful way to proceed – if the time and budget are allocated to make it possible.  As the GAC guide says about baseline data (I have added  emphasis):

"When should it be collected? 

Baseline data should be collected before project implementation. Ideally, this would be undertaken during project design. However, if this is not possible, baseline data must be collected as part of the inception stage of project implementation in order to ensure that the data collected corresponds to the situation at the start of the project, not later. The inception stage is the period immediately following the signature of the agreement, and before the submission of the Project Implementation Plan (or equivalent). "[p. 60]

In a rational process this would in fact be the situation.  But the reality is that for projects funded by GAC and many other donors, after two or three years of project design and approval processes, both the donor and the partners in the field want to start actual operations quickly.  

The amount of time allocated by donors and partners for the inception field trips by implementing agencies – and the budget allocated to support baseline data collection processes - are both too limited to make baseline data collection for all indicators during the inception period feasible in all but the most unusual cases.

A typical inception field trip for an inception period might last 3-4 weeks, rarely longer, and during this period

• a theory of change process has to be initiated with all of the major stakeholders, 
• an existing logic model tested and perhaps revised, 
• a detailed work breakdown structure, and risk framework developed, 
• institutional cooperation agreements negotiated, and 
• detailed discussions on a Performance Measurement Framework with a multitude of potential stakeholders completed.  

As the GAC guide states:

"As with the logic model, the performance measurement framework should be developed and/or assessed in a participatory fashion with the inclusion of local partners, intermediaries, beneficiaries and other stakeholders, and relevant Global Affairs Canada staff." [p. 57]

Some of these indicator discussions alone, where an initial orientation is required, and where there are multiple stakeholders, with different perspectives and different areas of expertise involved, can take 20 or 30 professional staff of a partner agency one or even two weeks in full time sessions, to reach initial agreement on what are sometimes 30 or 40 indicators.  

In some cases baseline data are available immediately, and that is one important criterion in choosing between what may be equally valid indicators.  

But in many cases, the data collection must be assigned to the partner agencies in the field, who know where the information is, and how to get it.  All of this means that a second round of discussions must be undertaken, to discard those indicators for which baseline data are unavailable, or just too difficult to collect and agree on new indicators. And, as the GAC guide quite correctly notes: 

"The process of identifying and formulating indicators may lead you to adjust your outcome and output statements. Ensure any changes made to these statements in the performance measurement framework are reflected in the logic model."  [p. 81]

The partners, meanwhile, have their existing work to continue with – and rarely see the baseline data collection as their most important operational priority given the political and institutional realities they face, as they do their normal work.

I have participated in several design and inception missions, and I can't remember when baseline data for all indicators were actually collected before the project commenced. At mid-term in many projects it is not unusual for an audit of the indicators by a monitor to find that 30-40%  of the indicators may not have baseline data, even after two or three years of project operation.

GAC itself appears to find it difficult to get baseline data for all of its agency-wide indicators, as I will discuss in the conclusion to this article.

All of this could be avoided if more money and more time – up to six months perhaps – were allocated to the inception period, with an emphasis on establishing a workable monitoring and evaluation structure, and actually funding baseline data collection.  

That means that when a donor agency emphasizes participatory development of indicators, during an inception period, it should be prepared to provide the resources of time and money necessary to make this practical.

Back to Table of Contents

2. Limiting the Logic Model to three levels

The GAC logic model has three results levels - for short-term, medium term and very long term changes. This is about standard for most agencies.  

But, of course, only two of these levels are actually operational, and susceptible to direct intervention during the life of the project – Immediate Outcomes in the short term (1-3 years on a 5 year project) and Intermediate Outcomes which should be achieved by the end of the project.  The Ultimate Outcome level is the result to which the project, along with a host of other external agencies, including the national government, and other donors, may be contributing.

In real life, a Logic Model which actually reflects the series of interventions, from changes in understanding, which are necessary for a change in attitudes, to changes in decisions or policies and changes in behaviour or professional practice, will go through a minimum of 4 to 5 or even more stages where needs assessments, and training of trainers or researchers lie at the beginning of the process, before we get to field implementation of new policies or innovations.

A logic model approximating the reality of project implementation would look like these examples from the Institute for Community Health in Massachusetts

Multi-level logic models from the Institute for Community Health

I have worked with partners in the field during project design where during the theory of change analysis, up to 8 different levels were identified, with assumptions, interventions and purported cause and effect links between these levels, before ever getting to the ultimate long-term result.  

It is impractical, the donors would argue, to have even a 5-level Logic Model – and this would indeed require extra work on indicators.   

The Global Affairs RBM guide does give a nod, on page 48, to the idea of “nested logic models”  This is something I have worked on with partners, and these can sometimes be more complicated to present and to understand than a 4 or 5-layer Logic Model.  If this guide is to be updated in 2022 it would be useful to see a more detailed illustration of how the nested logic models could be used.   

Some partners have decided to maintain their own more detailed, multi-level logic models, and present a simplified version to the donors. The whole purpose of these tools, after all, is not primarily for reporting to donors – but to help managers determine what interventions are working, and what changes are needed, before they report.  

And that is why the process is called Results-Based Management, and not just results-based reporting.  

These partners,  when an evaluator, a Minister, or a new donor representative has difficulty seeing how a simple two layer Logic Model can actually attribute results to interventions, have additional tools to use.  Having produced a more detailed and informal Logic Model for their own purposes, they can produce the real, multi-level Logic Model and explain the relationships.  

Even better, if they have a complete, and perhaps revised Theory of Change, they can explain what the Logic Model summary does not show, about alternative paths towards results, about the assumptions underlying the links between activities and results, and  about the risks that have to be dealt with if results are to be achieved.

It is unlikely that any donor will agree to a 4 or 5-level Logic Model, but it would be useful, as this Guide will eventually be revised, to include a section illustrating the process of nesting Logic Models, and if possible in 2021, to link each project's theory of change clearly to the simplistic Logic Models, so the limitations of the Logic Models can be seen clearly.

Back to Table of Contents

Is the GAC RBM system adaptive?

The CIDA/GAC approach to predictive planning for bilateral aid projects - specifying results and indicators in advance, and insisting that results are reported against those indicators, has been criticised for being too rigid, and "old school" for complex environments where problems change, and results may also need to change.   

As early as 2002, an internal Policy Branch review on the application of CIDA's RBM policy to Sector-Wide Approaches made the point that the CIDA Performance Measurement Framework  "is normally seen as "a monitoring and control tool imposed by CIDA, for its own benefit." 

In the Logic Model it is almost impossible to make changes to GAC project Intermediate Outcomes, once a project has been approved.  These are the results expected, in theory, to be achieved by project end.  For these and Ultimate Outcome changes, the process is ultimately so difficult that in almost all cases it has not been worth pursuing, until now. 

For more expensive projects (usually over $10 million)  the responsible Cabinet Ministers in both Canada and the partner country must agree to the Intermediate Outcomes, for a project agreement to be concluded.  For GAC, it is at this level that the scope of a project is determined - and what resources are required.  

If GAC officials think that a change in Intermediate Outcomes will mean a change in scope, and either a decrease in the amount of money justified for a project, or an increase in the resources required then the same level of approval, Ministers or other very senior officials on both sides,  must agree also to changes in those Intermediate Outcomes.  

For all practical purposes, given the many other pressures at the political level in both countries, that is rarely going to happen.

The use of templates for Logic Models which restricts the number of results layers, and the number of Intermediate Outcomes is convenient for agency managers who must aggregate results across a large number of projects, but this reinforces the argument that the RBM tools are designed less to assist in project management, than they are for senior management's agency-wide standardized reporting.

Operational Flexibility

This does not, however, account for the flexibility in GAC-funded projects at the operational level where projects are actually implemented.  

Immediate Outcomes can be changed during implementation, and for less expensive projects, Intermediate Outcomes can be revised to suit needs, because the approval process for projects under $10 million does not require Ministerial approval.

   As the GAC RBM guide says of the Logic Model and Performance Measurement Framework:

"They are iterative tools that can and should be adjusted as required during implementation as part of ongoing management for results."

Changes can be made to Immediate Outcomes if the reasons are documented and approved during the Annual work planning processes by Project Steering Committees, on which are local GAC officers, the implementing agency and partners, and representatives of the partner government.   This is not that difficult to do, if the situation warrants it. =

As I note in the final section of this review, GAC itself has changed its own agency-wide results

And there are other areas in which the GAC results system is more responsive to change than may appear on the surface.  

Indicator changes

Indicators define what results mean, and these can be changed if a case is made, and a project steering committee, of mid-level officials in both countries, agree that new indicators are better, more practical, or define changing results more accurately than the original indicators.

Beyond this,  in some cases it is evident that evaluations may not be based on the Logic Model or the Indicators originally specified, which means, in effect, that GAC is looking to see how a project evolved, and what results occurred, whether originally specified, or not.  

This, combined with the ability to make changes to Immediate Outcomes, suggest that there is some room for adaptive results-based management, "learning from results information and then practicing adaptive management to reflect what has been learned"  as John Mayne,  in  UNFPA's 2019 Results-based management principles and standards: The 3+5 Framework for Self-Assessment, suggested should be the practice.

One example of this potential for flexibility in how GAC applies RBM came in a May 2020 GAC request for proposals for an evaluation of a 6-year, $10 million scholarship fund for African scholars working in public administration, jointly funded with the MasterCard Foundation. 

The original GAC call for proposals for organizations to manage the project implied that the GAC RBM standards would apply.

Original GAC Call for Proposals Criteria

The project design apparently expected that not only would the scholars graduate from public administration programmes, and get work placements for mentoring, but that by the end of the 6 years they would have, in the words of the original Logic Model Intermediate Outcomes "Increased effectiveness [sic]...to contribute to public administration policy development in key sectors..." and "increased effectiveness...to provide leadership" in public administration and policy networks. 

While increased knowledge in the field should be possible after degree study, it is unlikely that new leadership qualities would be evident in 5 or 6 years.  

Evidently assessing the original project design as unrealistically ambitious, the  Terms of Reference for this evaluation state quite clearly that the logic model should not be used as the basis for evaluation, but that, essentially a retrospective theory of change should be constructed.  

GAC evaluation criteria - real results vs expected results

These evaluation Terms of Reference provide more detail [my emphasis, in blue, added]:

"The evaluation must follow the OECD/DAC (2010) Quality Standards for Development Evaluation and best practices in evaluation. Though, references are made to the OECD/DAC criteria (effectiveness, efficiency, relevance and sustainability), the evaluation must not be structured on the basis of the OECD/DAC criteria but rather on the basis of the evaluation questions indicated in section 3.....

 Although the implementation of the ... program has been guided by a logic model, it is not appropriate for evaluation purposes. The Consultant must review the intervention logic and construct an explicit causal Theory of Change (ToC). Thus, the analysis of the program’s theory of change, and the participatory update and validation of its intervention logic, will therefore play a central role in the design of the evaluation (inception phase), in the analysis of the data collected throughout its course, in the reporting of findings, and in the development of conclusions and of relevant and practical recommendations."

It appears that the implementing agency did not itself change the Logic Model or the indicators.  Had it done so, this would be a positive sign of adaptive results-based management. 

It is unclear therefore if this open-ended evaluation was just a hasty move to compensate for problems recognized too late in project implementation to be dealt with by the project implementation team itself amending the theory of change, logic model or indicators  - or  if it marks a genuine shift to a more adaptive form of RBM by GAC.   

But this, combined with the long-standing GAC flexibility on changing results indicators, suggests that there is considerable room for adaptive planning  and management in how the RBM tools are implemented, in practice, if managers pay attention to the process.

Back to Table of Contents

A Weak Agency-Wide GAC Corporate Results Framework

The current Liberal government was elected in September 2015, and the earliest results framework it could have influenced would have been that for the 2016-17 fiscal year.  

In July 2016 the Canadian government issued its Policy on Results, which among other things mandated the production for every government department of a Departmental Results Framework "that sets out the department’s Core Responsibilities, Departmental Results, and Departmental Result Indicators". That policy made it clear what a Departmental Result should look like:

"Departmental Results represent the changes departments seek to influence. Departmental Results are often outside departments’ immediate control, but they should be influenced by Program-level outcomes."

Notably, that policy did not specify a format which included either a theory of change or a Logic Model, but simply a general narrative.   And it did not establish standards for indicators aside from a definition: "A factor or variable that provides a valid and reliable means to measure or describe progress on a Departmental Result."

Ironically, for an agency which requires even small civil society organizations to produce a Logic Model and a workable performance measurement framework, there is no clear public evidence that GAC itself is aggregating results in any coherent way from Intermediate Outcome-level development results - the results which can be expected at the end of a project, or in GAC's case a series of projects worth roughly C $4 billion.

This is a change from past practice. CIDA, when it was an independent aid agency in 2006 developed a coherent agenda for Aid Effectiveness.

The CIDA Agency Aid Effectiveness Agenda framework in 2006 (click to enlarge)

This was followed by a new Corporate Logic Model in  the 2009 CIDA Business Process Roadmap 4.0, archived, fortunately by ReliefWeb.  Global Affairs Canada itself does not make it easy to find old policy documents.

The former CIDA Corporate Logic Model

A civil society organization bidding on or implementing a project worth $200,000-$300,000 or even much smaller projects, will be expected to produce a logic model, showing the contribution of short-term results to longer term results.  But there is no publicly available evidence of any coherent theory of change, or even a logic model for Global Affairs Canada in general, nor specifically for its development programming.  

Both of these may exist somewhere, but if so, they are not easily accessible by the public.  Neither the GAC Results-Based Management Page, nor the GAC departmental results reporting page include Logic Models or a Theory of Change.

Global Affairs Canada does not have, then, the kind of Corporate Results Framework that agencies such as the Asian Development Bank, or the African Development Bank have developed, and some (but not all) UN agencies have, making it clear how project level results relate to country programmes, some wider agency-wide framework, and ultimately to the 2030 Agenda for Sustainable Development.  

GAC is not alone in this among bilateral donors.  DFID, which did have such a framework, lost it when the development agency was fully absorbed within the Foreign, Commonwealth and Development Office in 2020, and it is difficult to find a coherent bilateral development agency results framework anywhere.

The OECD 2018 Peer Review referenced a developing attempt by GAC to relate project level results to agency-wide results, in an "Architecture for  Results of International Assistance". In October 2021 OECD published a short summary of why the framework, referred to with the acronym ARIA is needed, the complexities of developing it and an illustration of how GAC project results could be related to programme results and agency-wide results.

The developing GAC "Architecture for Results of International Assistance", OECD, October 2021

As of June, 2022, this framework cannot be found among any of the publicly available documents on the GAC website.

Nor can detailed project results be found online.  The  2022 Aid Transparency Index Report, published in August 2022, notes that Global Affairs Canada dropped from a "very good" score in 2021 to "good", with results reporting being the worst component of its score.

GAC ranking on the Aid Transparency Index 2022

"The performance component was Global Affairs Canada’s worst component. Results data samples failed; in most cases results were either missing or out of date. We could not find pre-project impact appraisals in Global Affairs Canada’s IATI data or published in other formats. Reviews and evaluations were only partially published"

The report goes on to recommend that GAC 

"....should also update and publish actual results data for projects that have been running for several years.

Global Affairs Canada should aim to disclose reviews and evaluations for all relevant activities. "

Table of contents

Changing Departmental Indicators

The GAC results reporting page, and the departmental plan do have a list of indicators for each of the 4 areas in which GAC works:  

• Diplomacy and Advocacy
• Trade and Investment
• Development, Peace and Security
• Help for Canadians Abroad
• Support for Canada's Presence Abroad

None of these are presented in a manner which demonstrates their contribution to an over-arching long-term result, which GAC itself might call an Ultimate Outcome.  

With the exception of Development Peace and Security, where the aid programme is housed, each of the other areas has 3-4 activity or results statements such as 

• "Canada's leadership on global issues contributes to a just and inclusive world"
• "Canada helps to build and safeguard an open and inclusive rules-based global trading system"
• "Canadian exporters and innovators are successful in their international business development efforts"
• "Canadians have timely access to information and services that keeps them safer abroad"
• "Personnel are safe, missions are more secure and government and partner assets and information are protected."

The results statements for the development programme are the strongest in GAC's list, actually using the phrasing outlined in the RBM guides:

• "Improved physical, social and economic well-being for the poorest and most vulnerable, particularly for women and girls, in countries where Canada engages."
• "Enhanced empowerment and rights for women and girls in countries where Canada engages"
• "Reduced suffering and increased human dignity in communities experiencing humanitarian crises"
• "Improved peace and security in countries and regions where Canada engages"
• "Canada’s international assistance is made more effective by leveraging diverse partnerships, innovation, and experimentation." (This one would not pass the GAC test, because it combines an activity and a result.)

CIDA, as the development agency was formerly named, in 2011 and 2012 contracted studies on the consistency of  Logic Model results and indicators in selected agency  priority themes.  Those studies, which included only a sample of projects, concluded  that while  many individual projects had Outputs, Immediate, Intermediate and Ultimate Outcomes consistent with agency priorities, indicator consistency was weak, dealing primarily with Outputs.

Many of the indicators used today for the development results at the agency level - at least as it is publicly presented on the GAC website - are still essentially in 2021 indicators for Outputs, or at best Immediate Outcomes, not indicators for longer term changes.  For example (my comments in italics):

•  "Number of graduates (m/f) of GAC supported, demand driven, technical and vocational education and training."

• This is perhaps an Immediate Outcome indicator, given that GAC is apparently monitoring the number graduating, but without a Logic Model telling us what activities contributed to capacity development in technical and vocational schools, and improved recruiting of students, it is really difficult to say for sure what this indicator represents.  A more interesting indicator of the longer-term effectiveness of the training would be the percent of graduates getting work.

• "Number of women’s organizations and women’s networks advancing women’s rights and gender equality that receive GAC support for programming and/or institutional strengthening."   - - 

• The question here is - what did they do with the money, and what changes in women's rights or in organizational effectiveness have occurred?  As it stands, and in the absence of a Logic Model, this appears to simply be an indicator of an activity - not even an Output.

•   It is also distressing to see that this new indicator replaced a meaningful indicator from the GAC 2018-2019 results report: "Percentage of countries that demonstrate an increase or positive change in women’s access and control over property, financial services, inheritance, natural resources and technology.

• That useful indicator was itself new in 2018, but did not survive the year.

• "Number of entrepreneurs, farmers and smallholders (m/f) provided with financial and/or business development services through GAC-funded projects"  

• This is essentially an Output indicator. Of more interest would be an indicator related to income, or the sustainability of the farming, after the entrepreneurs and farmers were provided with money and capacity development training.

Occasionally, an indicator does appear which provides more information on the result of activities.  For example: 

            "Percentage of countries that show a decrease in the adolescent fertility rate (number of births/1000 women)." 

 But there are not enough of these solid indicators, and the indicators listed in the Departmental Results Report would not, for the most part, meet the indicator assessment criteria GAC itself sets for its development partners: 

• "Do the indicators at the ultimate level measure the change in state, condition or wellbeing of the ultimate beneficiaries described in the ultimate outcome statement?
• Do the indicators at the intermediate level measure the changes in behavior, practices or performance of the intermediaries or beneficiaries described in the intermediate outcome statement?
• Do the indicators at the immediate level measure the changes in capacities, such as skills, knowledge, abilities, etc., of intermediaries or beneficiaries, described in the immediate outcome statement?
• Do the output-level indicators measure the existence or quality of the products or services for which the implementer is accountable or the process by which those products and services were produced?"

 Applying these indicator assessment criteria I could find, with the most generous assessment, only 4 of 15 listed indicators in the 2019-2020 GAC results report for the development programme which would meet the GAC project-level criteria of indicators for Immediate or Intermediate Outcomes.    

An argument countering this might be made that it is not possible to report longer-term results because 11 of the 15  indicators presented on the results reporting page for 2019-2020 are new indicators, created in 2019. In  this case, anything beyond Outputs is unlikely to be seen.  

• That would be more convincing if a review of the previous years' (2018-19) results indicators didn't show that all of the 13 indicators for that year were new
• Only 5 of those new indicators in 2018-2019 were carried over to 2019-2020.  
• It stands to reason that if enough indicators are replaced every year, there will never be enough good indicators of Immediate or Intermediate Outcome level results to demonstrate change of the kind GAC results are supposed to represent

If we go back to the first year of the current government's mandate we see three legitimately useful indicators for agency wide results,  which could provide a useful picture of progress if continued over time.  All of these indicators were discarded after just two years.

3 Useful GAC Agency-wide indicators which were discarded - click to enlarge

And in breach of GAC's own stated policies on providing baseline data - only 13 of 15 indicators for 2019-2020 had either a target or baseline data when the 2019-20 results report was produced.  Three of the remaining 12 might be excluded because they were responding to changing and (perhaps) unpredictable humanitarian crises, but that still leaves 10 other indicators without baseline data, a year after the indicators were created.

• One explanation of this is that Covid interrupted the data collection, but these baseline data should, in theory have been collected by April 2019 when the 2019-20 fiscal year began, and before the pandemic.  

• And in any case, a look at the  indicators  for the previous year (2018-19) reveals that of  eleven indicators which were not dealing with humanitarian issues, 10, again, did not have baseline data.  Covid could not have interfered with data collection in 2018.

• Another puzzling explanation for the lack of baseline data for some indicators is listed in a footnote:  "Baseline information was not available when the targets were established in the 2019-20 Departmental Plan. Targets have since been established for these indicators."  

• A basic premise of indicator development at the project level is that no reasonable targets can be established without a baseline.

Given that project-level indicators are designed to demonstrate results at the Intermediate Outcome level, analyzing what percentage of projects actually achieved their results - using valid and meaningful indicators of change - would tell us how successful GAC development activities are overall, and what progress is being made over time. 

At least one of the other sections of the results report, that dealing with international advocacy and diplomacy, does indeed provide such information on the department's results in that area. 

Reviewing the GAC results reports  from 2016-2017 to 2019-20, and the departmental plan for 2020-2021, it is possible to see how frequently GAC has changed indicators for development results.   

• All three indicators in 2016-17 were new, and had no baseline data.  
• Those three continued to be used in 2017-18, and therefore produced some useful comparable data on results.  
• But all three disappeared, and were replaced by 13 new indicators in 2018-19.  
• 2019-2020 produced another 7 new indicators.
• By the time the 2020-2021 Departmental Plan was produced, the number of  indicators with baseline data increased substantially, if we take the previous year's results as the baseline.  This demonstrates just one of the merits of using the same indicators over time.

A more generous assessment of the extent to which the indicators actually measured change might be possible - but only if accompanied by a Logic Model, showing us what the relationship is between GAC activities, short-term, mid-term and long-term results.  

Readers may wish to consult the GAC results reports listed at the end of this article, to make their own assessment or look at the 2023 review of GAC results reporting by the Auditor-General of Canada.  But it is worth noting that since this review was first published in November 2020, GAC results reports and work plans have stopped referring to baseline information.  

Patterns of changes in results statements and indicators for Global Affairs Canada reporting 2016-2021 - click to enlarge

There is no doubt that GAC funds many potentially worthwhile activities, but agency-level reports should give us not just descriptions of activities, and the occasional almost anecdotal reference to isolated indicator data, but information on results.   If the agency wants to meet the same standards on results reporting that recipients of project funding must meet,  GAC should be telling us not just what it did - but what changed. GAC reports should consider, for example :

• Not just how much the agency spent on education, for example, but how many students trained by these programmes got jobs;

• Not just how much was spent on preventing gender-based violence, but whether rates of violence decreased.

Aggregating this kind of information will not be particularly easy,  but that is true at the project level for managers receiving GAC grants too, when they have to report on results.  Aggregating data on change could tell us what percent of the projects GAC funds met targets about improvements in a variety of areas, such as improvements to income, jobs, health, reduction in gender based violence, for example.

The frustrating thing in reading this type of GAC report is that in the text above the indicators there are examples of data for individual projects which could contribute to better indicators for agency-wide results.  For example,

• A project supporting women's economic empowerment in Kenya, reported that its support to small and medium enterprises had increased revenues by 53% for SMEs including by 48% for small enterprises, and increased employment by overall by 28% and increased employment in small enterprises by 56%.
• In Tanzania, five political parties adopted a  manifesto  on women's participation in politics in part developed by organizations supported by GAC, and added key women’s issues to their party platforms for the October 2020 national elections.
• Another project in Tanzania, in part supported by GAC Canada’s $3.5-million contribution averted 377 maternal deaths, 175,982 unintended pregnancies and 43,828 unsafe abortions.
• A project focused on emergency maternal, newborn and child health care in Rwanda. contributed to a  52% reduction in maternal mortality during childbirth and a 36% reduction in neonatal mortality due to prematurity.

Aggregating more data like this would produce indicators more meaningful than those currently found in the GAC development indicators.   The fact that this is not being done suggests perhaps, that 12 years after the earlier study of  GAC project indicators, many projects being funded still do no use indicators which describe real results, as opposed to simply completed activities.

Since I originally reviewed this new reports have been published by GAC.   In the 2022-2023 GAC plan for the Peace and Development program there are 15 department-wide indicators, only 3 of which could, by GAC’s own RBM standards be considered true results indicators at the Outcome level. The rest are indicators for Outputs, or completed activities, something GAC itself will not accept as true indicators for results from those agencies it funds. And of those 15 indicators 8 have no baseline data, which would normally be used to demonstrate results – measuring change from the starting point.

GAC's own agency-wide results report for its development activities provides a narrative of activities and occasional reference to indicators, which would not be acceptable in any development project it funds.

A reader her might say "Well,  Armstrong, the last time you look at this was in 2021, and things may have changed by now."   But the March 28, 2023 Auditor General's report to Parliament  on  GAC programming between 2017 and 2022 came to much the same conclusion as I have here.

"Global Affairs Canada established indicators to monitor

progress toward achieving policy goals. However, we found that

24 of 26 of the department’s policy indicators did not measure

outcomes. An example of this was the number of people reached with

nutritional support, which is not an outcome but an output. An outcome

would be improved health status or increased life expectancy. The

department’s reports contained information that accurately represented

some of the department’s work but did not capture the full range of

outcomes.

4.23 We assessed 60 projects to determine whether the department

demonstrated that it tracked policy indicators. We found that the

department only used 35 of these projects to report on policy goals.

In our opinion, reporting that omits almost half of the projects is

incomplete.

4.24 In 2015, Canada adopted the United Nations’ 2030 Agenda

for Sustainable Development and committed to advancing the United

Nations’ Sustainable Development Goals in Canada and abroad.

Progress against many of these goals is advanced by international

assistance activities. The department mapped the policy indicators to

the related Sustainable Development Goals so that it could measure

project contributions toward each goal. Specifically, Goal 1 relates to

poverty reduction and Goal 5 relates to gender equality.

4.25 Because the department was not using its policy indicators

to capture outcomes for many of the projects, it was also not tracking

contributions to Sustainable Development Goals. As a result, the

department missed an opportunity to demonstrate the value of

international assistance.""

             Source:  Auditor-General of Canada, Report 4 - Global Affairs Canada

The Bottom Line

The new Global Affairs Canada Results-Based Management guides are  necessary tools for anyone designing or managing Canadian aid projects.  These guides also provide a practical, very useful set of resources for anyone who wants clarity on the process of results-based project design, management or results reporting, funded by any agency.

There is little evidence, however, that the attention to results GAC clearly aims for at the project level, translates into any kind of agency-wide, coherent corporate results framework. 

Table of Contents

Further Reading:  Other Early CIDA/GAC RBM  documents  (organized by date)

• Peter Morgan.  The Design and Use of Capacity Development Indicators (1997)
• CIDA.  The Logical Framework:  Making it Results-Oriented (1997)
• Results- Based Management Division, CIDA. Lessons Learned from Implementing Results- Based Management (RBM) in CIDA (1998)
• Mark Schacter. Results-Based Management and Multilateral Programming at CIDA: A Discussion Paper (1999)
• CIDA.  Overview of the Bilateral Project Cycle - chapter 5, p. 9 11- Logical Framework Approach, and chapter 9, p 18-21. (Feb. 1999).
• Réal Lavergne.  Results-Based Management and Accountability for Enhanced Aid Effectiveness. (July, 2002)
• Evaluation Division, CIDA.  CIDA Evaluation Guide. 2004

GAC Results Reports

• GAC Departmental Results Report 2017-2018 
• GAC Departmental Results Report 2018-2019
• GAC Departmental Results Report 2019-2020
• GAC Departmental Plan 2020-2021

Auditor-General of Canada

• Report 4: International Assistance in Support of Gender Equality, March 2023

_______________________________________________________

Back to Table of Contents 

GREG ARMSTRONG

Greg Armstrong is a Results-Based Management specialist who focuses on the use of clear language in RBM training, and in the creation of usable planning, monitoring and reporting frameworks.  For links to more Results-Based Management Handbooks and Guides, go to the RBM Training website.