Consultative Impact Monitoring of Policy - CoIMPact - Example: Kenya's Participatory Impact Monitoring - KePIM

	Background of KePIM As part of the process of implementing the Poverty Reduction Strategy Paper (PRSP) within the framework of the National Poverty Eradication Plan (NPEP), the Government of Kenya, with the support of the Social Policy Advisory Services project of the German Development Corporation, GTZ, are in the process of undertaking a participatory impact monitoring exercise, known as KePIM. The lead agencies from the government’s side are the Human Resources and Social Services Department (HRSSD) and the Central Bureau of Statistics (CBS) in the Ministry of Finance and Planning, and the Poverty Eradication Unit (PEU) in the Office of the President. KePIM, Kenya’s Participatory Impact Monitoring exercise is designed to assess the effectiveness of various poverty focused policies and programmes. In particular, it is designed to monitor progress in the implementation of the PRSP. It uses non-standard participatory methods of data collection to elicit the knowledge, views and opinions of local people. Among the tools used in the first round of the exercise (and in other similar exercises) are resources maps, time lines, Venn diagrams, focus group discussions and matrices for the analysis of the data. Exercises that generate “qualitative” data offer poor people’s perceptions on their quality of life, as well as their priorities, constraints and opportunities for improving their situations. This type of data deepens the understanding of poverty, and helps to define the priorities of the poor for poverty reduction interventions. KePIM is a process, not a one-off-exercise, and will be carried out over a number of rounds, building on the knowledge and experience gained in earlier rounds. To date one full round of data collection has been carried out in six districts in 2001 (Kwale, Guvha, Makueni, Vihiga, Transmara and Mandera), and a second round of investigation, in eight of the country’s districts, is planned for 2002 (Murangaa, Butere -Mumias, Bomet, Kisumu, Nairobi, Garissa, Mwingi and Malindi).
	Objectives of KePIM The first and second rounds of the exercise were carried out in five phases: First the preparatory phase established the institutional framework for the exercise, including finding a consensus on the objective, purpose and results of the exercise. Second phase involved undertaking a policy analysis in order to determine the extent to which the priorities of the poor have been integrated into the Poverty Reduction Strategy and Sectoral Policies. Third phase involved the collection of data using participatory methodologies. The fourth phase involved the analysis of the results and the production of a report. The fifth phase involved the dissemination of the results of the exercise as well as the follow-up of the recommendations made in the report, to ensure that the voices of the poor are incorporated into Government policy. The overall objective of the KePIM exercise was to consult with the poorest sections of the community to ensure that their voices and concerns are included in the on-going policy making process. It is expected that KePIM will eventually become an institutionalised qualitative monitoring system using participatory methods, involving both government and other stakeholders to keep policy makers informed about progress in implementing and assessing the impact of the PRSP. It is one of the objectives of the exercise that the capacity of government, and its partners in the development process, is established so that they can carry out future rounds of the exercise on their own. Figure 1: Main phases required for one round of the KePIM exercise 6
	Policy Areas for Investigation The KePIM exercise has strived to ensure a link to policy – this starts with the identification of areas for investigation, through the analysis of previous reports and documentation, and then undertaking a detailed policy analysis of the areas to be investigated. This strong policy focus is one of the major differences between KePIM and a traditional Participatory Poverty Assessment. The first round of KePIM assessed at the impact of policies in the areas of health, education, water and sanitation and food security. After the completion of the analysis phase it became apparent that two very important areas required further detailed investigation – the delivery of agricultural extension services and people’s access to credit. These were taken up in the second round of KePIM. The following checklist can be adapted for use for the policy analysis work: What are the available resources in the village or community? What is the individual or group’s definition of poverty? It is reasonable to expect that this will be different for each group. It is vitally important to record, accurately and honestly, what is said. What are the Causes of Poverty, in order of importance? The reason for ranking the causes is to give an indication of what areas in the future would be the most suitable for action. Are there specific intra-household elements of Poverty? This question is trying to find out who suffers most in the household - whether this is the head of the household, women, children, the elderly. The local attributes of wealth groups? The community should identify how many wealth groups there are in the community. Current stratification of the community. This should try and allocate the number of people in the community into the previously identified groups, using various ranking exercises. Changes in the stratification of the community. There will have been changes in the number of people living in the various groups identified in the previous question, over the past three to five years. Causes of Changes in the Poverty Situation. The causes of the various changes, whether for good or bad, should be recorded and probed. How have those effected coped with the changes in their poverty situation? This addresses the survival strategies that people undertake to deal with problems they face, such as not having enough food, or loosing their job.
	Selection of Sites In the first round of the exercise, 18 sites were visited across six district – in the second round, it is expected that 16 sites will be visited in eight districts – providing a little more breadth to the study. The individual districts to be researched have been selected using a detailed, purposive, selection process to ensure a broad variety of livelihood systems will be investigated. This included the development of a composite poverty indicator to take account of issues of income, access to water and education, while also including aspects such as the population density. Feedback Loops Those involved in KePIM have always stressed the importance of the link between the exercise and policy formulation. This is based on the assumption that for a PRS (or any other policy) to be successful, effective and relevant, it requires feedback that can be used to make adjustments to interventions and provide information on future policy decisions. To ensure this can happen a conducive institutional framework was established at the start to make sure that senior policy makers and planners were on board from the beginning. The feedback process includes: A briefing session with each of the communities visited, to ensure that they have an opportunity to validate the findings and correct anything they see as wrong. Discussion and presentation of findings at the district level after the visit of the data collection teams. Feeding the findings of the report back to key national level decision makers. A number of reports have been produced in order to facilitate the feedback process. These include the full final report; a popular version of the report, and a series of briefing leaflets that highlighted the key findings and recommendations from the report. The briefing leaflets were produced in non-technical language as well as in Kiswahili.
	The KePIM Tool Box: Participatory data collection exercises, such as KePIM, collect different types of data, these can be grouped under the following headings: Temporal data: is related to time. The tools that provide this kind of information include timelines, trend lines and seasonal calendars. Spatial data: is related to the location of the community. This data can be generated from sketch maps and transect walks. Institutional data: relates to the access of the respondents to the institutions that surround them, or are supposed to serve them. These include venn diagrams.´ Some tools act as Eye Openers. These assist the research team in gaining an insight into the community without the need for deeper analysis – in general they are not contentious and can include maps, transect walks, timelines and calendars. The second group is more Analytical in nature and help the research team to get a deeper understanding of the topic under discussion, included under this heading are wealth rankings and problem analysis.
	Ten guiding principles for participatory data collection: Change from doing to facilitating, from teaching and telling to listening. Communities have local knowledge, all the research team does is tap into this and organising it. The secret is to focus on shared learning. Be Open Minded and do not bring in preconceived ideas. It is also important not to jump to conclusions. Remember the research team are the visitors and the local knowledge of the “villagers” is equally important and reliable as any scientific knowledge. Seek out the poor; the research team has to make a particular effort to find the poor and the vulnerable to ensure that they get a chance to speak. Every situation is different and has to be treated as such . In this case the team needs to use their Own Best Judgement - at times different approaches may be needed with different groups to get the desired information. Share Information not just amongst members of the team but with the community, meaning the community gains control over the information and the exercise moves from being extractive to being more empowering. Feedback to the communities is vital—there must be a final meeting where all the important insights gained are fed back to them for their comments and clarification and possibilities for action (from either side) need to be identified and discussed. Optimal Ignorance means only generating the necessary information. The secret to the success of this is “knowing what I don’t need to know”. Triangulate everything and cross-check all information given by looking at it from various perspectives. This can involve applying different techniques, utilising secondary information collected elsewhere, asking different groups or key informants the same questions. Triangulation helps to achieve greater accuracy in the information, and helps to give credibility to the results. Visualisation allows all participants to see, comment, adapt and alter the responses being given, it is also a means of cross-checking. Visual techniques provide scope for creativity and encourage a frank exchange of views – they also allow illiterate members of the community to participate. Total Immersion in the life of the community helps to improve the quality of the information collected and offers the opportunity to break down the formal barriers, which will persist between outsiders and the communities. It also affords the opportunity to observe the life of the community outside office hours.
	1. Semi Structured Interview In depth, Semi-Structured Interviews (SSI) can be conducted with individuals or groups. The SSI helps in the establishment of a dialogue between the interview partners. An interview checklist (or guideline) is used which serves as a guide during the interview and ensures that comparable information is generated from a number of people. The checklist determines only the interview topics and the respondent is encouraged to expand on their answers, allowing a degree of freedom. 2. Time Line The Time Line shows major events in the community over a certain time period. The time line can deal with issues such as the seasonal planting of crops, or major interactions with the government (such as elections or aid initiatives). The events are displayed along a line drawn on paper (or on the ground) starting with the most distant event, and marking or symbolizing subsequent events up to the present. It presents the views of members of the community on what events have affected them mostly or were most important to them. 3. Well -Being Matrix (Wealth Ranking) The Well Being Matrix can show how different groups perceive their (or their community’s) social and economic status. Criteria for social and economic status are identified, as well as different socio-economic levels existing in the community. The end objective is to show how many people (or households) belong to each social group. 4. Venn Diagram A Venn Diagram (also known as a Chapatti Diagram) captures the perception of the communities on the social and economic relations that impact on them. The community (or individual) is drawn in the middle of the chart. The institutions with which they interact are represented as circles arrayed around the community. The size of the different circles surrounding the interview partner represents the importance of the institution to them, while the position of the circle (close or distant) shows how often the interview partner has contact with the mentioned institution. This shows the working relationship between the respondents and the institutions. 5. Service Map Following on the Venn (Chapatti) Diagram, the community can be asked to draw a Service Map to identify the actual distances they must travel to access services. For the sake of consistency, the services included here should be those identified on the Venn Diagram. 6. Village Resource Map The village map is designed to show the location of the various households in the village in relation to the local infrastructure (such as roads), resources (such as water) and various amenities (such as churches or government offices). This tool is generally non-sensitive, and can be used in a large group as an icebreaker. 7. Transect Walk (Line) A transect walk (or line) is a cross-sectional picture of the physical or geographical diversity of a community. It shows the main land features, uses and variations. It provides the opportunity to validate and crosscheck information collected using other methods. It also allows researchers to develop a clear impression of the physical community. The exercise should be done in the company of a small group of local people who will be able to explain to the team what they see. 8. Pairwise Ranking This tool can be used to distinguish a group’s priorities or preferences, for instance problems encountered or services available . It starts with the community identifying their own priorities / problems, and then devising a matrix that will allow for them to be compared to each other, one pair at a time. 9. Problem Ranking Problem ranking helps in identifying people’s priorities with regard to problems or needs. In general, the facilitator asks the group for their problems and lists them during the discussion. The interviewers are asked to “vote on” or rank the problems according to their importance by placing markers (for example stones, beans, paper clips) on each “problem”. The more markers a problem receives, the higher the priority for solving it. By giving each person some physical symbol (i.e. paper clip, stone, bean) it helps to ensure that those without a voice get the opportunity to present their point of view and to prioritise the identified problems jointly. 10. Problem Analysis Flow Diagram A problem analysis diagram shows a selected “main” problem in the centre of the diagram and draws links from the different causes of the problem through to the effects that this has. This tool can be especially strong if the interview partners can identify solutions for each aspect of the diagram, particularly if it is an area they can take action on themselves. The Problem Analysis Diagram is designed to provide more detailed information about a selected priority problem, and to trace its cause-and-effect relationship. It also allows the possibility for identifying potential solutions to the problems. 11. Calendars (Seasonal and Daily) Calendars, whether seasonal or daily, can be very valuable tools for assessing changes in availability of goods or experience of problems throughout the years, or the daily schedules of women. It gives researchers insight into the schedules for both men and women and also allows an easy comparison of the schedules. Furthermore, when schedules are compared with each other researchers are able to determine the amount of productive time available.
	References GTZ-SPAS, Ministry of Finance and Planning, HRSSD, 2002. “An introduction to KePIM, Kenya’s Participatory Impact Monitoring Exercise.” GTZ-SPAS, Ministry of Finance and Planning, HRSSD, Special Issue August 2002. “An introduction to KePIM, Kenya’s Participatory Impact Monitoring Exercise.” GTZ-SPAS, Ministry of Finance and Planning, HRSSD 2002. “KePIM Key Findings”.” GTZ-SPAS, Ministry of Finance and Planning, HRSSD, August 2002. “An introduction to CiReCa, Kenya’s Citizen Report Card Initiative” Ministry of Finance and Planning, HRSSD, 2002. “Kenya Participatory Impact Monitoring: Field Manual.” Pain, Chris. Undated. “Stepwise Approach for the Implementation of the Kenya Participatory Impact Monitoring Exercise.” Wagle S and P Shah (2002) “Report Cards: A General Note on Methodology” World Bank, Washington.