1. How the research was born

In 1985, the Italian Parliament passed a special act establishing an important financial fund destined for a considerable Italian aid to the economically most backward countries in the world, where famine and de-nutrition were flooding. Such a fond was named Fondo Aiuti Italiani (FAI) . It has been managed by the Ministry of Foreign Affairs, by creating an ad-hoc entity, supposed to be agile and fast, given the state of urgency and emergency provisions required by the critical situation of this type in numerous areas of the world

Therefore, the FAI has urgently selected some countries in the need of help, most of them African, and some sectors of intervention, elaborating several relevant projects of intervention. At the same time, the decision-makers of FAI, since the beginning, were concerned that if the interventions couldn't be decided following a more general programming of them altogether, nevertheless, it was necessary – in a very short time – to become conscious of the ‘effects' and results of such interventions. The decision-makers of FAI entrusted the Planning Studies Centre (by convention of 1986) with the task to study and propose a system of permanent evaluation of the FAI programmes and to inform the public in general and the operators of FAI in particular about the outcomes of such programmes.

 

2. The task of the research

The task of the study groups of the Planning Studies Centre has therefore been to study:

•  An evaluation system of FAI interventions , which could be capable of giving a clear order to the objective system of FAI itself, in the absence of which it is difficult to define which strategic criteria should measure the effectiveness of the intervention itself and, therefore, its evaluation;

•  A systemisation of the national intervention projects (already decided to be urgent within the objective system of FAI) concatenating objectives and the very projects in a ‘programme structure' to which always refers the effectiveness of the projects themselves;

•  To fix a system of indicators or result measurers adherent to the programme structuring, capable of letting us evaluate the achievements of the objectives;

•  And gradually down into the concatenated objective/instruments system, to build a method capable of taking under control the implementation of the programmes and of evaluating their capacity to be in line with the objectives, and of measuring definitively the effectiveness and the congruity with the cost and expenditure that were implied.

The Pro-FAI research has permitted the Planning Studies Centre to improve a method of strategic programming adapted to this case; and it has permitted the Ministry of Foreign Affairs to constitute a permanent system of external evaluation of the different implementation projects of the FAI programmes.

 

3. The final research report

The Planning Studies Centre concluded its research in the course of six months (from September 1986 to February 1987) with a final report, entitled: Methods and Processes of Evaluation of the FAI Programmes (see general index in the publication section).

This report consists of seven volumes, of which we will here, very briefly, give the contents extracted from the general presentation of the Report by the director of the Centre Prof. Franco Archibugi.

The first question that the study group of the centre had to face has been that of intending and consequently defining to which kind of ‘evaluation' it was possible to refer, given the vast gamma of ongoing approaches in different international and national circles and in the scientific community itself.

‘However, largely adopted by the scientific community and encoded at the level of the United Nations in more than one official document, the general principle was followed that ‘ the evaluation is that process which tries to determine – as systematically and objectively as possible – the relevance, the effectiveness and the impact of any activity in the light of its goals and objectives' .

On the basis of this principle, from the constitutive acts of FAI and from the first official decisions which determined it, and with the cooperation of the competent authorities of the FAI management, has been extracted that system of goals and objectives in the light of which the evaluation itself should be proposed (the subject, however, is that included in the general introduction to the report in the first volume ).

The methodology proposed for the evaluation of the FAI programmes is illustrated in the first two volumes of the Report (a first application of it to certain FAI Projects is illustrated in the third volume ).

In the course of the research, however, the study group has met, in this country, a general state of acquisition concerning the conception, the experience and the procedures connected to the theme of evaluation, something rather unsatisfying with respect to the experience and progress accomplished in other countries and in the international context, especially that of the United Nations. This has induced the study group to consider in an extensive way the mission received and to outline also some possible procedures oriented at the installation of a permanent system of evaluation, including both the ‘programmes' and also individual ‘projects', and for the whole so-called ‘project cycle' from its initial formulation to its ex-post evaluation, passing through all the intermediate phases of monitoring and evaluation. (This system of procedures, amply modelled based on those that were recommended at an international level, most of all at the organs of the United Nations, is the object of the fourth volume of the Report).

Finally, as conclusion of the development of the methodological research, the study Group has been obliged to acquire the final data of the entire expenditure of FAI (as at the time available); and it has been considered opportune to supply a classification of available data according to expenditure categories oriented at a ‘system of objectives' which was studied in a ‘methodological' phase: one more way to get a general assessment on the entire expenditure and its destinations. At that occasion then it has also been tried to get a first general assessment of that expenditure without following, of course, the criteria, parameters and indicators of the general, proposed methodology, because of the insufficiency of data, means and time, but satisfying oneself with some very approximate first evaluations. (The results of these two ‘accompanying' works of the research to the principal task of the study group are in the fifth and sixth volume of the final Report).

In performing the research, the study group has acquired a great amount of scientific and technical documentation. The study Group has been obliged to perform a selection of the material, which went by instead of an evaluation of it. It seemed that this heritage of the research, information and selection should not be dispersed, but could be encompassed at the benefit of those who are (have been or will be) involved in the FAI activities, and more generally in the activities of the cooperation for the development. (It is thus born the seventh volume (as Appendix) of the Report, which constitutes a ‘Guide' to the scientific and technical documentation on the methods of programme evaluation in the less developed countries.

The presentation of the seven volumes of this Report (of 1986), on behalf of the Centre's Director, is closed with the following consideration which has not lost its validity or worth after 20 years:

‘Let us quickly display – at the closure of this general presentation – the spirit in which the Planning Studies Centre, a ‘non-profit' research association devoted to the methodological research on planning, has undertaken and carried out this research on hand. For the Centre, this has been an important occasion in order to make a step forward concerning its own research programmes: a step forward concerning the methodology of evaluation, which is an essential point of a good evaluation. But not only associate researchers of the Centre should be interested; our true hope was, and, in this case, also remains that we have contributed in making evaluation a process integrated to the ordinary management of programmes and activities, at least in the so delicate and debated field of cooperation, inside the Italian administration. Thus in the spirit that inspired the General Secretary of the United Nations, Javier Pèrez de Cuellar, when the presentation of the book ‘Manual of Evaluation' ended, prepared by the UN secretariat and addressed to the operators of all UN systems: ‘ I'm expecting from anyone of you an engagement in incorporating the self-evaluation as integrated part of the decisional and management process '.

The seven volumes of the Pro-FAI Research Report are available as ad-hoc reprints (unfortunately, only in Italian) among the publications of the Planning Studies Centre (see here).