google-site-verification: googlefccb558c724d02c7.html

Translate

Tuesday, August 15, 2017

Evaluability Assessments and Results-Based Management: 8 Guides

by Greg Armstrong

Evaluability Assessments answer the question:  Is there enough available information to justify the time and cost of a full-scale evaluation? If they are done early enough during implementation, they can identify basic problems in design which can guide remedial action.

One old study set the stage for 7 recent detailed Guides which synthesize earlier work and provide useful advice on whether, when and how to undertake an evaluability assessment – or to use other approaches to assessing project or programme design integrity.

Evaluability Assessment Guides
Evaluability Assessment Guides



Level of difficulty: Moderate
Length: 3-72 pages
Primarily useful for: Evaluation managers, RBM specialists doing evaluation assessments
Most useful: DFID Guide to Planning Evaluability Assessments
Most interesting:  Program Management and the Federal Evaluator (1974)
Limitations:  These guides tell us what needs to be done, but they require people with the process skills to do it all.


The History of Evaluability Assessments


Evaluability assessments, pre-assessments or exploratory evaluations, as they were known in some cases 40 years ago, have been used for many years in public health research where the term “evaluable” refers to ‘Patients whose response to a treatment can be measured because enough information has been collected”,  education,  and justice  (PDF) and the term has academic antecedents in testing mathematical propositions, far back into the 19th Century. 

The Foundation Evaluability Assessment Document


The foundation evaluability concepts for social programmes were initially presented, from what I can see in a 1974 Urban Institute study, one of many emerging during implementation studies of the Johnson administration's Great Society Program,  published as "Program Management and the Federal Evaluator" in the Public Administration Review, included in a 1977 volume of Readings in Evaluation Research, available through Google Books. 
Title page of "Program Management and the Federal Evaluator"
The Foundation Evaluability Assessment article
It advocated looking at issues which are at the heart of current guides to evaluability assessment - and for that matter, at the heart of solid results-based design:
  • Whether there is a clearly defined problem addressed in design
  • If the intervention is clearly defined
  • If the short and longer term results are defined clearly enough to be measurable
  • if “the logic of assumptions inking expenditure of resources, the implementation of a program intervention , the immediate outcome to be caused by that intervention, and the resulting impact” are “specified or understood clearly enough to permit testing them”.
  • Whether managers are capable of and motivated to using performance data for concrete management decisions.
While there are many other guides to evaluability assessment which are much more detailed of more practical utility today, the most important core concepts are here in this 1974 study, and this short, 43-year old article is still worth reading for its insights on clarity on language, assumptions and results.  
Leonard Rudman’s later 1980 book Planning Useful Evaluations - Evaluability Assessment  set out a detailed approach to dealing with all of these issues.

Changes in Evaluability Assessment Utilization


Evaluability Assessments started out as ways of improving the design of evaluations, to increase the chances they will be useful to the people who are funding and implementing activities, and in recent years, with a substantial increase in the number of evaluations of international development projects, many agencies, including UNICEF, the World Bank, UNDP and numerous bilateral donors have incorporated evaluability assessments as part of the project cycle, after design, during implementation and before a decision is made to pay for a full-scale evaluation.  

Many Evaluability Assessment Guides have been produced in recent years, but here are just a few worth noting.

RBM Training

RBM Training
Results-Based Management

Share on LinkedIn

Subscribe to this blog by email

Enter your email address:

Delivered by FeedBurner

 
Read the latest posts