Evaluating capacity development in research and development organisations: towards more effective capacity-development efforts

The term 'capacity development' is highly elastic, in that it can be stretched and shaped to cover a variety of different elements including training, formal education, networking, technical assistance and investments in physical and scientific facilities. In many cases, capacity-development efforts involve multi-faceted organisational strengthening programs.
Capacity development is defined as an ongoing process by which individuals, groups, organisations and societies increase their abilities to perform core functions, solve problems, define and achieve objectives, and deal with their development needs in a broad context and sustainable manner. Organisational capacity development refers to an ongoing process by which an organisation increases its ability to formulate and achieve relevant objectives. It involves strengthening both its operational and adaptive capacities. While an organisation may benefit from external support, ultimately it is responsible for the development of its own capacities.
Such capacity-development efforts are fundamentally different from programs that provide well-defined products and services for relatively uniform populations of users. For this reason, different approaches are needed to evaluate them. There have been few systematic evaluations of capacity development strategies, to test their underlying theories and assumptions, to document their results or to draw lessons for improving future programs. One reason for the lack of systematic evaluation is that organisational capacity development processes are inherently complex and dynamic. The familiar difficulties of evaluating the development of individual capacities are also greatly compounded at the organisational level.

This project aimed to advance thinking and practice in the design, management and evaluation of organisational capacity development in the context of international development. It focused on issues at the intermediate, or meso, level of organisations and organisational arrangements.

The collaborators adopted an action-research process of identifying key problems, clarifying concepts and issues related to capacity development, field-testing approaches and methods for evaluating capacity development and drawing substantive and methodological lessons from the results. The project's work was oriented by five 'Guiding Questions':
What are the key abilities or capacities that need to be developed in research and development organisations?
By what process(es) does organisational capacity development take place?
How can external agents/agencies contribute to organisational capacity development
How should organisational capacity-development efforts be evaluated?
How can evaluation contribute to capacity-development efforts?
The project collaborators undertook six evaluation studies during 2000-2001. In July 2001, a Midterm Workshop reviewed the studies to draw preliminary conclusions and plan future project activities.
In 2002, members of the ISNAR coordination team visited each of the organisations where the evaluations were carried out, to gain a deeper understanding of the evaluation processes, to participate in the analysis of results and to aid in the dissemination and use of the evaluation findings. In May, a workshop was held to draw conclusions from the six evaluation studies in relation to the project's guiding questions and to prepare materials for a book highlighting the case for evaluating capacity development.

Preparation of the book became the major outcome of the study. A publication committee guided its preparation. The materials prepared in the workshop were revised, complemented and edited by the ISNAR Coordination Team and a professional writer.
An outline for the book was proposed by ISNAR, and participants at the May 2002 workshop were asked to prepare written responses to seven sets of questions, based on the review of the six evaluation studies:
What do you believe are the key capacities that need to be developed in research and development organisations and why?
How do you believe organisational capacity development takes place?
What kinds of contributions can external agencies make to organisational capacity development and how should they go about it?
What methods should be used to evaluate organisational capacity development and how should they be used?
How have the evaluations been useful? What and how have they contributed to (a) organisational capacity development or (b) organisational performance? What other uses have the evaluations had?
What are the key institutional issues related to (a) organisational capacity development, and (b) evaluating organisational capacity development?
How adequate are the theories of action presented in the evaluation studies? What are your suggestions for improving them?
These topics corresponded roughly to the chapter titles of the proposed book. A complete manuscript was reviewed by several specialists in capacity development and evaluation. In early 2003 the publication committee decided on final revisions to the manuscript, and substantive changes were made over the next two months. ISNAR's Publication Unit arranged for final editing of the book, titled Evaluating Capacity Development: Experiences with Research and Development Organizations around the World, and it was printed in August 2003.
Several publications presenting results of the project were prepared in English, French and Spanish. Both the ECD book and the summary pieces were posted on various web sites. Presentations were also made to various professional groups. A proposal to support the Use of Evaluation in Institutional Learning and Change has been prepared, based in part on results of the ECD Project, and has received initial support from the Rockefeller Foundation, the International Fund for Agricultural Development and several CGIAR centres and partner organisations.
Given the time and resources required for producing the book, it was not possible to prepare the anticipated guidelines for evaluating organisational capacity development.