Measuring Impact in the Age of the Data Revolution
- 20 april 2018
As a student in development economics I have had courses related to impact measurement, and how to use econometrics to analyze the data that we gather. The conventional approach is based on gathering data before and after project implementation, via surveys, to statistically measure what impact was caused by the project. The Impact Measurement & Knowledge team that I am working with at Oxfam Novib also bases its research activities on this approach. However, the data world around us is changing. It is estimated that every minute the world sends 15 million text messages, logs 3.6 million Google searches and posts half a million tweets. Several sources predict exponential data growth toward 2020 and beyond, with a doubling of the size of the digital universe every two years. Given these massive new sources of data, do new possibilities arise for the practice of impact measurement in the development and humanitarian sectors?
During Oxfam Novib’s annual Expert Meeting, organized on the 29th of March 2018, successful examples of how the data revolution can be used in development and humanitarian settings were introduced. The aim of the day was to address how impact measurement and monitoring and evaluation (M&E) in general can make use of these new data sources.
Stefan Hoevenaar stressing the importance of testing your innovation with the target population along the process of innovation.
The day kicked-off with an inspiring talk by Stefan Hoevenaar, founder and creative director of ILUMY Digital Innovation and Plek, about the process of innovation in general. Organized around three tracks (1: Using innovative sources of data; 2: Human-centered design and user-centric approaches; and 3: Data in the field), we could hear examples directly from experts working with the data revolution in their own sectors. All sessions together steered the closure discussion of whether we can take the plunge regarding the data revolution and impact measurement and M&E.
Throughout the day it became clear that overall the data revolution offers promising opportunities for the development and humanitarian sector. One example that was given was the ‘Your Word Counts’ project of Oxfam Great Britain; a mobile case management tool which captures, analyses and responds to feedback from people receiving humanitarian assistance. This tool should give those affected by crisis voice for improved support and services and can also be used for gathering data around sensitive topics. Another example focused on the use of (big) data for optimizing the worldwide flow of food to hungry people. Furthermore, the Chatbot presented by HumanityX offers promising opportunities for scale and efficiency in the humanitarian context. This text processing technology facilitates direct communication with affected populations in humanitarian crisis situations. It classifies text data and uses this to provide real-time information. See also the post of the UNHCR Innovation Service on chatbots in humanitarian settings.
How about Oxfam Novib? Are we innovating based on the opportunities triggered by the data revolution? Currently, Oxfam Novib is working on an innovation project called Datalab. This project implements a “two-way data street”; it provides data-driven services to target citizens, while simultaneously collecting data on the usage of the service to help target communities in need and evaluate the impacts of program and projects. An example of Datalab in practice is the household budgeting tool implemented in Uganda. This tool helps users manage their own household spending and saving to build assets and escape poverty, whilst collecting data on how often people use the app and whether people reach their saving goals. The information collected can be used for example to inform advocacy campaigns or for designing financial literacy trainings. The human-centered design of Datalab aims to empower people with actionable information and creates digital space.
These are all promising examples of using the data revolution in the development and humanitarian sector. The question left is whether we can also take the plunge regarding impact measurement and M&E. Personally I see opportunities for amongst others collecting anonymized data on sensitive issues by using technologies such as mobile management tools and the Chatbot. People might be more eager to share about issues such as intra-household violence when they do not have a person in front of them, but just a digital device instead. Another opportunity exists for speeding up the data collection and analysis process by innovations such as Datalab: when providing real-time dashboards, citizens as well as Oxfam can directly act on insights provided by the dashboard, such as which fertilizer leads to highest yield performance. Also data collection via drones and satellites might speed up baseline data collection and the monitoring work for projects related to land use, human settlements or deforestation.
However, major challenges remain, such as being inclusive when using technologies and avoiding self-selection related to big data (i.e. not everyone has access to digital devices). Another challenge is responsibly storing and protecting citizen-provided data, and ensuring that we track changes caused only by the project intervention itself. Nevertheless, I believe the data revolution can benefit impact measurement and M&E activities. The future will tell whether the now-standard approach of collecting data via surveys will become outdated, but personally I believe it is not necessarily about replacement of one practice for the other. Rather, new data sources and techniques from the data innovations can be an added value to how impact measurement and M&E is currently practiced. The Expert Meeting itself pointed towards an essential factor for making this all a success: collaboration among different sectors.
Je aanmelding wordt verwerkt
Welkom bij de club!
Wat fantastisch dat je je hebt aangemeld als vrijwilliger! Samen kunnen we armoede verslaan en zorgen voor een rechtvaardige wereld. Houd je mail goed in de gaten voor de eerst volgende acties waar jij bij kunt helpen.