| dc.description.abstract | The consultant team found five monitoring and evaluation functions in PLUS: (1) to validate technology or interventions, (2) to track project outputs, (3) to show project and intervention impact in economic and environmental terms, (4) to elicit farmer perceptions and foster their participation for understanding and evaluation, and (5) to support strategic analyses that further project implementation. Our goal is to suggest improvements for all these functions.
SECID has given strong leadership, particularly in monitoring interventions. PADF and CARE have always supported reporting outputs. After some initial concern because of the amount of effort required to implement the system, both implementing agencies now support M&E and advocate its continuing implementation. Both have hired M&E staff at all levels of the project. There is a spirit of innovation and a willingness to try M&E techniques.
The M&E case studies of interventions (hedgerows, rockwalls, gully plugs, and gardens) have been a major effort and the basis for most of SECID's analysis of PLUS interventions. They were designed to provide information on the interventions' effectiveness, costs and returns. They estimate incremental differences in production and income by comparing fields subject to interventions with matched fields using traditional practices on "witness" plots. The data on cost of establishing the technology and maintaining it are from frequent visits to interview the farmers who are involved. Harvest data are obtained by interview or by harvesting sample areas in the treated fields and control fields. The studies are limited to the small monitored areas, with 50 installations on somewhat fewer farms.
The current M&E system has also been following interventions in several zones chosen as representative. The agencies have mapped the plots in these zones.
The case studies and monitored zones feed data to calculate a set of indicators called SPIs. Implementation has been spotty, with data problems for most indicators. For impact estimation, the results of the case studies have been generalized to all adopters, an expedient method that is becoming less appropriate as the number of participating farmers increases. Staff of the implementing agencies recognize the difficulties in using csae study data to estimate impact.
Our general principles for modifying the current M&E system are the following:
The M&E should be integrated with PLUS project implementation and relevant to the needs and decisions of farmers, field staff, mid-level managers, participating NGOs, implementing organization managers, and donors. In particular, farmers and farm families should be increasingly active in PLUS M&E using participatory techniques. M&E information should be relevant, clear, accurate, and representative, and it should cover the main impacts of the project every year. The M&E indicators should be reported yearly in a uniform manner, put in time series, compared with goals, disseminated, and discussed. Beyond annual reporting, M&E data should be analyzed using statistical and geographical tools to support strategic decisions. M&E information should be focused, reliable, and economical.
Recommendations for Impact Estimates
The project has grown to the point where it is time to estimate impact in a more representative manner. Fortunately, each of the implementing agencies has developed lists of participating farmers with their basic characteristics. PADF maintains file cabinets of dossiers, while CARE has a computerized system. These files are already used for implementation and for counting project outputs. While these files were not developed for M&E, they are an invaluable resource that should be used fully to obtain representative and defensible estimates of project impact. We recommend that they be used as a sample frame to make such estimates.
An extensive survey (short interview, large sample) would be used to estimate gross environmental and economic impact and quality of implementation on farmers fields. The project may choose to implement additional intensive surveys (longer interview, smaller sample) for estimates of other kinds of impact and for quantitative strategic analyses. It may choose to use data from existing GIS data sets. Other low-cost methods are noted for providing socio-economic and environmental data for analysis: regional RRA/PRA methods or sampling localities for group interviews.
The project should estimate the number of secondary adopters, or those who adopt without direct contact with the project. It is possible that project impact is considerably larger than what would be estimated by surveying only participating farmers.
We recommend that the current case studies be continued through the end of 1995 so that they generate detailed data on costs and production. Thereafter, we recommend that the case studies be monitored using less intensive methods to generate time series of gross production and environmental impact.
The economic analysis of the interventions should include representative assessment of gross incremental production, continuing calculation of financial indicators, and rough calculations of two new products: intervention budgets (showing investment, costs, and returns) that can help regional and field staff and approximate whole-farm budgets for typical farmers.
Recommendations for Farmer Participation
We recommend continuation and expansion of the farmer participatory methods already being adopted by PLUS. We recommend a continuation of the use of FSRE Diagnostics or Rapid Rural Appraisals but recommend evolution in the direction of a Participatory Rural Appraisal approach (PRA), with the aim of involving farmers themselves in identification of key constraints and opportunities. We recommend that these Diagnostics, together with other information sources, become the basis for the development of comprehensive descriptions of target-group farms, including the development of whole-farm budgets. We also recommend the adoption of Participatory Monitoring and Evaluation (PME) into future PLUS activities. This will involve the organization of special Farmer Evaluation Sessions (FES) so that farmers, themselves, may be engaged somewhat more formally in the assessment and evaluation of the PLUS interventions.
We recommend that efforts be made to involve regional staff--especially M&E staff--more extensively in the analytic work of the M&E system. The M&E system should support them to do regionally-specific analyses, and that they play a key role in the preparation and use of both intervention-specific and whole-farm budgets.
We recommend that PLUS initiate a dialogue with the local NGOs and farmer groups with which it works to stimulate them to give leadership toward the preparation of a comprehensive plan for the continuation of those elements of PLUS that are essential to make the PLUS efforts and hillside agriculture in Haiti more sustainable.
Recommendations for Land Husbandry Monitoring
Implementation of the PLUS goal and purpose requires the adoption of a "holistic" better land husbandry approach: the care and management of the land for productive purposes.
The impact of the project on the bio-physical environment should be monitored at three levels:
- the individual plot/micro level
- the farm household level
- the macro-geographic or project area level
We recommend that the project compile baseline data sets on the bio-physical conditions within each of the localities where the project has field activities from existing secondary data sources and the knowledge of key local informants. This information should be used for defining and demarcating onto a topographic base map individual land management units for all of the project areas. These LMUs should be used as the basis for stratifying environmental impact and adoption rates of the different project interventions.
The overall conservation effectiveness of farmers land use/management practices should be used as the basic criteria for determining the environmental impact of the project interventions. For instance concern with better land husbandry means that how the land is used between the hedgerows and rock walls, or in the catchment area upstream of a plugged gully, needs to be assessed to determine the overall environmental impact and conservation effectiveness of field level improvements.
Quantitative assessments of the conservation effectiveness of individual interventions in the context in which they are adopted should (in combination with the use of simple visual indicators of the status, type and severity of soil erosion) form the basis for arriving at an overall better land husbandry rating. Its main purpose with regard to the PLUS M&E system would be to provide a clear indication as to the extent with which the land use management practices on a particular plot, individual farm holding or over a wider geographic area conform to the principles and practice of better land husbandry. If they do, then they would be in line with the projects goal and purpose and could be used as an indicator of success.
We offer specific guidelines for implementing this concept of land husbandry in the PLUS ME system.
Recommendations on SPIs
The individual SPIs are discusses in this report. We recommend that the SPIs, the output indicators from the project log frame, and a limited number of new indicators be combined and reported in an annual PLUS ME Impact Report.
The Products of the M&E System
The main activities of the PLUS ME system would be,
Participatory Rural Appraisals
Farmer Evaluation Sessions
Farmer Dossiers and Lists
Surrounding Areas Sample Frame
Extensive Survey
Land Husbandry Status Reports
Optional Intensive Surveys
Output Implementation Reports
The reports generated by the PLUS ME system should be integrated into the agencies annual reporting practices in a way that reduces overlap. The elements are as follow,
Annaul PLUS M&E and Impact Report for the project as a whole, with the following elements
Output indicators from both implementing organizations
Process indicators of participation, including summaries of area FES, PRA, and staff discussions
Impact indicators for the project as a whole
Intervention-specific results, such as trials, case studies, FES
Appendices: FES, PRAs, technical reports, special studies
Occasional reports on technology, adoption, and impact of each project intervention, with a minimum of one report per intervention over the course of the project.
Mid-term and final impact evaluations
Participatory rural appraisals, with a minimum of one per area.
Annual M&E report from each project area, including a tabulation of some data items, a summary of the Farmer Evaluation Session, a Land Husbandry Status Report, and text by the area project team.
M&E reports TO each area annually, with intervention budgets, tabulations of area data, responses to queries, and comparative data.
Special studies and analyses, minimally to cover the strategic questions on achieving impact raised by project staff. | en_US |
