Impact Matters

The World Citizens Panel is an impact measurement method developed by Oxfam Novib. We combine quantitative research (impact surveys) with qualitative research (stories of change) to give participants in our programmes a voice, to learn how our programmes can be improved, and to contribute to the public debate on effectiveness of development cooperation. In 2012, the World Citizens Panel started in Uganda and we currently work on impact measurement in Somalia, Pakistan, Mali, Cambodia and Nigeria with more countries to come.

Oxfam Novib started the World Citizens Panel in 2011. The initial toolkit and its website were designed by Butterfly Works. The impact measurement approach was based on earlier work by Oxfam Novib and partners in Cambodia, India, Occupied Palestinian Territories, South-Africa, Bolivia, Ethiopia, Kenya, Senegal and Rwanda. These partners have developed and tested a set of key impact indicators and questions.

Nowadays, we work in several countries in Africa, Asia and the Middle East.

Why measure your impact?

Impact studies enable you to get a better understanding of the change processes that you intend to stimulate, to report about impact in a consistent way, and to define and implement follow–up actions which improve the interventions. This allows you to look ahead and plan strategically in a fast changing context.

Scoping studies and measuring impact contributes to evidence based resource management. In addition, impact studies allow you to be accountable for the contribution to a more sustainable world without poverty to the people that participate in your programmes as well as to donors.

Our impact services

Oxfam Novib makes impact management accessible for all organisations through a cost-effective and transparent impact measurement methodology. We provide 2 tailored products:

  1. Scoping Report

The Scoping Report is the end product of our scoping service. This service allows organisations to identify the most pressing problems of their target group according to the people themselves. Stakeholders (implementing organisations and (prospective) beneficiaries of an intervention) are central in our approach: together they identify priorities for the new project. Knowing the key needs and challenges for life improvement of the target group can result in interventions that have a greater impact. To identify these needs and challenges, we use the Stories of Change method in combination with a quantitative Survey. The results are summarised in a Scoping Report.

  1. Impact Report

We measure the impact of an intervention by analysing the changes in the lives of people living in poverty and inequality over a longer period of time. To identify if changes have occurred, we use an Impact Survey (preferably a baseline and an endline survey to allow comparison over time) among the target group and a control group. The Stories of Change give a better understanding why and how these changes have happened. Based on the results, managers can define and implement a follow-up strategy to improve their efforts.

Our Impact Services share a number of key principles:

  • Giving citizens (as primary stakeholders) a voice in interventions
  • Involving staff, partners and managers in the process in order to enhance learning, strategic reflection and follow up
  • Keeping costs at a minimum

Mixed methods approach

Oxfam Novib uses a balanced approach of qualitative and quantitative methods with a focus on changes in people’s lives. We make these changes visible in an empowering, inclusive and rigorous way. The methodology consists of quantitative Impact Surveys and qualitative Stories of Change.

Impact Surveys include:

  • Facilitation of a kick-off workshop for implementing organisations to prepare the surveys and define indicators (based on a Theory of Change)
  • Use of a standard questionnaire plus tailored questions for each specific intervention
  • Random selection of people from the target group and a control group
  • Training of interviewers (staff of implementing organisation or people specifically hired for this work) how to collect high quality data
  • Provision of a smartphone app and/or online questionnaire to collect the data
  • Impact calculations based on statistical methods. The results are summarised in an Impact Report
  • Facilitation of a reflection workshop on the results to facilitate learning, stimulate programme improvements and define areas for further research

Stories of Change include:

  • Facilitation of a kick-off workshop for implementing organisations to define the domains for the study
  • Training of implementing organisations on the Stories of Change methodology
  • Facilitation of the stories collection. These are registered on paper or in short videos
  • Participatory analysis of the collected Stories of Change
  • Summary of the findings in an Impact Report
  • Facilitation of a reflection workshop on the results to facilitate learning, stimulate programme improvements and define areas for further research

Although we consider both the Impact Surveys and Stories of Change valuable methods on their own, we prefer a mixed methods approach to measure impact. A combination of both methods can capitalize on the respective strengths of each approach. Pairing quantitative and qualitative components of a larger study can achieve various aims, including corroborating findings, generating more complete data and using results from one method to enhance insights attained by the complementary method.

Research team

Oxfam Novib’s research team is driven to deliver high-quality study reports. Our team has excellent research skills, is experienced with coordinating research studies in complex contexts and has extensive knowledge of measuring social and economic changes in people’s lives.

Our team is dedicated to help others to identify new priorities, to develop more effective interventions and to contribute to external debates. This includes supporting managers and implementing staff by providing advice, training and technical resources.

What drives us?

Oxfam Novib is driven to contribute to more effective poverty reduction. We are dedicated to build more inclusive and sustainable local societies through the delivery of inclusive impact measurement services to initiators of interventions to low income populations in developing countries. Our products and services are aimed to accelerate impact through the inclusion of low income community members.

Contact

Oxfam Novib provides impact measurement services to both Oxfam managers and their partners as well as to external clients such as donors, microfinance institutes, small and medium enterprises, NGOs, etcetera.

To find out more about what we offer, please send a message to: worldcitizenspanel@oxfamnovib.nl