Governance Assessments and Governance Indicators – The Governance of Forests Initiative Indicator Guidance Manual

By GregArmstrong

[Updated August 2019]

Don’t let the title mislead you - The Governance of Forests Initiative (GFI) Indicator Framework and the accompanying Guidance Manual published  in 2013 by the World Resources Institute are practical tools not just for those working on environmental governance, but for anyone developing indicators for the design, management or monitoring of governance  projects or programmes in any sector.

Level of Difficulty: Moderate to Complex
Length: 295 pages - or downloadable as 7 discrete files by component
Languages:  English
Primarily Useful For: Indicator workshop facilitators, project or programme managers
Most Useful:  Guidance on data collection for 596 indicators
Limitations: These are not cut and paste indicators.  The Guide provides examples of issues to be considered in data collection, but users will need to do their own work to adapt indicators to specific contexts



Background

Developing indicators for governance projects and programmes– whether democratic governance in general, or governance in specific sectors such as natural resources or the environment -  always requires careful discussion and debate among those designing and managing projects, and their partners and stakeholders.  .

There are usually very few shortcuts we can take in working through what potential indicators are at the same time both technically and politically valid, and practical enough to actually have collectable data.  In my experience on both environmental governance and democratic governance projects, it is necessary often to start from scratch on each unique project, clarifying assumptions about what the results mean to different groups, and what is likely to be practical in terms of data collection.  For projects on transparency, democratic governance, and accountability it is often counterproductive to simply adopt standard global indicators used to compare country performance on governance, and try to use them to describe specific capacity development needs of individual governance projects.

Useful publications from the World Resources Institute 

I was therefore pleasantly surprised to discover, belatedly, the World Resources Institute and its publications, particularly those on The Governance of Forests Initiative.  The WRI states that while it ensures that its publications meet academic standards, “We also ensure that all of our publications are timely, fit for audience, and rooted in a strategic plan for achieving positive change in the world.”  Based on my reading of the indicator guidance manual, this is certainly fit for the audience of people I work with – people designing, managing , monitoring ,evaluating, and affected by governance projects – in forestry, natural resources management or any other sector.

Practical governance indicator guidance

There are dozens of reports and guides on governance issues and governance indicators on the WRI website, but after sorting through many of them, what I found the most useful for facilitators of indicator development workshops for people working in the field on governance projects in a wide range of sectors is the series of publications on indicators for assessing forest governance.  These were field tested over several years in Cameroon, Indonesia and Brazil, before publication.

For those wanting just an overview of the process of assessing governance, the 68-page Assessing Forest Governance: The Governance of Forests Initiative Indicator Framework might be of interest.  It is, however, somewhat mislabeled I think, on the publications download page  as a “full report”.

There are numerous alternatives for downloading the indicator framework and reports, a summary or individual chapters at the WRI website.

Governance of Forest Initiatives Indicator framework download alternatives
[click to enlarge

The 295-page Governance of Forests Initiative (GFI) Guidance Manual: A Guide to Using the GFI Indicator Framework , written by Lauren Williams, Jessica Breitfeller and  Celine Lim, while not as pretty as the framework itself, contains, as far as I could see, both the report, and much more detailed discussion of the practical issues involved in actually using the indicators, or adapting them to project or community specific contexts. It is this document which I think will be of greatest use to people who need not just to develop governance indicators, but to specify how they will be defined and how data will be collected.

This Guidance manual is divided into two parts – an introductory section on how to conduct governance assessments, and detailed guides to collecting data on 596 indicators in part 2.  

While I think the whole Guide is worth downloading as one coherent document, the website also,  for those with limited bandwidth, interest or patience, provides links to 7 sections with theme-specific indicators which can be downloaded separately. The rest of this post describes what is in those sections, and in the Guide to using the indicator framework.

How to conduct a governance assessment

Principles of good governance

 This 35-page introductory Part 1 of the guide discusses the principles of good governance for which it is proposing indicators, including transparency, participation, accountability, coordination and “capacity”, and four forestry themes with which they will be matched in indicator development.  

Matching Governance Themes and Indicators
[Click to enlarge}

Those themes are forest tenure, land use, forest management, forest revenues.  But the Guide also recognizes the need for indicators on cross-cutting governance institutions and cross-cutting governance issues of relevance to any sector.

Building a Monitoring and Evaluation System: IFAD’s useful M&E RBM Guide

by Greg Armstrong

[Updated July 2018]

A “Guide for Project M&E”  produced in 2002 by the International Fund for Agricultural Development and written in clear language, still provides solid, practical, and actionable advice on the design, and implementation of a Monitoring and Evaluation system.  The RBM tools provided here, although replaced in new and updated guides, remain useful for project design, planning, monitoring, reporting or evaluation, in 2018 or beyond.

Level of Difficulty: Easy to Moderate

Length:378 pages

Languages:  English

Primarily Useful For: Planners, evaluators, monitors, managers

Most Useful:  “Navigating the Guide” identifies sections of use to different readers

Limitations:  Originally produced in English, French, Spanish and Arabic, this guide has been replaced by new and updated M&E guides, and only the English language version of this 2002 edition is easily available. 

Background

In November 2013 I worked briefly with IFAD Vietnam, helping them develop a simple reporting format for their projects.  I was struck then by the practical nature of the discussions, working with participants who were literally grounded in the realities of rural development, and with the interest IFAD staff take in developing workable tools for project management.

It was then that I first came across the IFAD  Guide  for Project M&E,  written by Irene Guijt and Jim Woodhill. It is something I regret not seeing when it was first produced in 2002. The fact that it was produced 12 years ago does not, however, detract from its ongoing utility.

Who This is for

While those getting the most obvious benefit from this RBM guide will be IFAD staff, partners and consultants working with them, the basic ideas and the clear organization of the document will make if useful for programme and project planners, monitors, evaluators, and  field staff working with any organization,who are trying to figure out how to make something practical out of the usual donor results-based management technical terms.  The primary target audience for the document is obviously going to be people who focus on agriculture, food production and associated activities, and while they will know a lot about those topics, they, like most people, will not want to wade through, nor are they likely to remember, pages of donor-specific RBM jargon.  

This 2002 Guide is written in relatively clear language, [and in 2018 I find it remains more user-friendly than the more recent guides] and while it does refer to some basic RBM terms, it is easy to get past this.    The terminology different donors use in Results-Based Management change with the winds of management fashion, but underlying the jargon, if you can find them, are some basically simple ideas.  This document presents these basic ideas well.

Indicator data visualizations: The Guardian Datablog and Data Store

                                                                                          

--Greg Armstrong--

“Facts are Sacred”, The Guardian Datablog says, and the Guardian challenges us to examine those facts carefully, as its recent story on the "3 Little Pigs" illustrates.

While creative visualizations of  indicator data undoubtedly make it more interesting, without informed explanation of what the information means, the visualizations can often be misleading.  The Guardian Data Store and the Guardian Datablog provide an example of how journalists bring images, indicator data and intelligent analysis together.

 [Edited to update links, June 2018]

Level of Difficulty: Moderate to Complex

Primarily Useful for: Aid agency communications managers, creative programme reporting staff

Most useful: The Development Data directory

Limitations:  The raw data searches work only intermittently, and the Kindle format reduces the utility of the Ebook.

Who this is for

Not everyone will be able to produce the kind of data visualizations presented by the Guardian, but we can all benefit by the examples the news site provides of where to get data, how to present it, and what kind of questions are necessary in determining the reliability and validity of indicator data.

Those who may have the resources to actually produce these kind of visualizations will probably not be working at a project level.  But aid agency communications staff, and some large agencies at the country level, may have the intellectual and financial resources to do what the Guardian data team does.

Background - data visualization

Many development workers, project managers, monitors, evaluators, wading through dense international development project reports and evaluations, may wish that more attention could be paid to making indicator data more understandable.

The late Hans Rosling, for example, is well known for creative presentations on indicators.

These are intended, as the Gapminder Institute  puts it, to “unveil the beauty of statistical time series by converting boring numbers into enjoyable, animated and interactive graphics”.  But doing this at the project and programme level in international development can be challenging.

And as entertaining as animations, graphs or pictures are, by themselves they  can be misleading.  Hans Rosling  does more than provide the graphics, of course. He interprets what the data and the graphics can tell us, in clear, compelling language.

The Gapminder Institute is not alone, however, in presenting – and interpreting - indicator data in a compelling manner.

The Guardian Data Post and Datablog

News websites  reach millions of people a day, and among these the most creative in obtaining and utilizing data in compelling visualizations is by far the The Guardian.  The Guardian newspaper’s online site  with more than 29 million unique visitors in December 2011 alone, is the fifth most visited newspaper, and among the most respected sites on the internet.

The  most interesting part of the Guardian site, (aside from 3 Little Pigs ad) for many of us working in international development is the Guardian Datablog, and the associated Data Store,  its directory of all of the statistics the Guardian uses as it reports the news.  This includes World Government data search, a Development data search, examples of featured data visualizations and a link to an electronic version of Facts are Sacred, a new book by Simon Rogers, one of the Guardian’s news editors, on how The Guardian collects and presents data,. He is also editor of both the Data Store and Datablog.

The Guardian itself, in its main economic, political, health or education pages publishes mainline news stories. What the Datablog does, as far as I can see, is highlight the stories making innovative use of publicly available data, explain where the data come from, and then challenge readers to question it, or do more with the information. In some cases the Guardian Datablog appears to produce its own visualizations from raw data sources, but in most, it seems, the Datablog team provide a link to, or a variation on another agency’s visualization – and then they provide The Guardian’s explanation of what the information means.

Scope of the Guardian’s data

The Manchester Guardian, one of the UK’s oldest newspapers, was founded in 1821 and established its online site almost 175 years later, in 1995. The Guardian Datablog was established in 2009 to explore what editor Simon Rogers refers to as data journalism – journalism which mines available data looking for hidden or emerging stories.

As he points out, data journalism is not new – good journalists have been using obscure data as the basis for breaking news for centuries.

The Datablog, between January 15, 2009, and the time it produced a summary of all of the blog’s comments on data journalism, on July 15, 2011, listed 1,407 articles or blog posts, including links to all of the available underlying data in spreadsheet format.  It is unclear why that information had not been updated since it was posted in 2011,  but by my very rough estimate as I write this in March 2012, there have been approximately 490 - 500 additional posts between August 2011 and February 28, 2012.  This brings the total to approximately 1,900 articles -  all with some form of graphic – tables, charts, static or interactive, simple or complex, and all challenging us to use the underlying data ourselves in whatever way makes sense to us.

This is a mind-boggling number of analyses, given the detail involved - over 10 such articles a week for two and half years – and it does not necessarily represent the true extent of the work The Guardian has done.

As just one example, one story, by The Guardian Data editor about Malaria, referenced in the Datablog on February 3, 2012, for example, was preceded and followed within a week, by at least three other stories on Malaria in the main section of the Guardian  (by the health editor), in the Guardian Weekly, and in the New Review section  of the Guardian’s sister publication the Observer.

Old data - new presentations