Guide to Evaluating Capacity Development Results

Author: 
World Bank Institute
Publisher: 
World Bank
Year of publication: 
2012

This set of guidance notes is designed to support practitioners and evaluators in conducting retrospective evaluations of a capacity development intervention or portfolio to assess and document results. Users will enhance their understanding of the capacity development process, of what works and what does not work in promoting change and to inform future programs.

The standard M&E approach for assess- ing capacity development results has not been sufficient. These guidance notes are designed to complement and supplement good M&E practice to more effectively identify capacity development results. Typi- cally, results-based M&E emphasizes the assessment of outcomes and impacts while 9also tracking inputs, activities and outputs to monitor implementation. A results chain or logic model is used to articulate the sequence from inputs to results.

For example, in World Bank lending operations, a project’s results framework specifies the project development objec- tive (PDO), higher-level outcomes that reflect the achievement of this objective, and intermediate outcomes that need to be in place to reach the desired results. M&E arrangements in the Bank’s project documents usually specify key outputs and how to track them during project imple- mentation.

The results chains for the capacity components of development projects often remain poorly defined for the following reasons:

  • The standard levels of indicators (such as PDO-level and intermediate outcomes) do not necessarily trace the achievement of capacity development objectives. The achievement of a capacity change objective is not the same as the achievement of a PDO. In many cases, institutional capacity changes are required as an intermediate outcome before a PDO can be achieved. Although the targeted capacity change process is key to the success of the overall project in contributing to the related development goal, the milestones needed for achieving this institutional capacity will be largely overlooked. Depending on the specific case, institutional capacity changes might be captured at either the PDO or intermediate levels, but the relevant intermediate capacity outcomes (ICOs) are rarely associated and tracked, creating a missed opportunity for learning about what worked and what did not for the capacity development interventions.
  • The role of change agents and the targeted change process(es) need to be identified. Capacity development entails preparing or empowering designated local change agents to initiate and/ or manage needed changes. What changes need to be measured cannot be determined without analyzing existing capacity constraints and specifying a capacity change strategy. Only then will it be possible to assign indicators for the desired institutional capacity changes and the ICOs.
  • Context matters. Standard sector indica- tors lack adaptability and assume that institutional arrangements have the same meaning in different contexts. A nuanced understanding of each capacity development change process is needed to identify which indicators are appropri- ate for assessing targeted institutional capacity changes and ICOs.

Overall, the use of the traditional results framework or logic model for assessing the achievement of a capacity change objec- tive too often leads to the problem of the “black box” of capacity development, wherein the needed improvements in the ability or disposition of stakeholders remain undefined and unmonitored.

This guide focuses on retrospective evaluation for two principal reasons: (1) prospective evaluations such as random- ized control trials are often impractical to implement in the context of capacity development interventions and (2) external evaluators are often called in after the fact to assess the results of an intervention. However, in many instances the topics and guidance apply to prospective evaluations and monitoring activities.