Jorge Romero Leon
Unit manager Learning, Innovation and Knowledge
We are after a moving target
When you are ambitious and aim to fight inequality at all levels, like Oxfam Novib does, you need very creative and effective impact measuring to inform your strategies. Understanding how social change happens and learning from what you are doing as you do it, requires rigorous research, effective monitoring and evaluation, and concrete and co-created knowledge. As Jorge Romero Leon puts it: “We need methods that help us capture the moving target we are after: systems transformation.“
Romero Leon, who joined Oxfam Novib in January 2023 after being a program director at Oxfam Mexico, acknowledges the importance of impact measurement. Yet he prefers to talk about measuring results instead of impact. “We usually don’t have the timeframe of at least 10 years necessary to measure the long-term, life-changing or system-changing outcomes that we call impact.”
“What we can measure are short-term and medium-term results that impact the lives of people and communities we work with directly, as well as results in the policy arena. At Oxfam Novib we look at connected results at the local, national and global levels, at the different factors that lead to changes at each level and at how these changes link up across levels. Then we can say, for example: how is the new Loss and Damage Fund (see page XX) likely to create changes at the local level when you put those resources to use?”
“One of our core values is inclusiveness. We are convinced that there is no way we can measure transformative change from the outside. We have to ask the communities we work with how inequality works for them, what they want to change, and how they want to measure that change, so they can use it in their own strategic thinking. Therefore we design and co-create our own methods; we have also adapted a much-used method like Outcome Harvesting, to enable colleagues and partners to autonomously monitor their progress towards system changes.”
“Participative results measuring demands a lot of work from our partners. Oxfam Novib supports technological innovations to help ease this burden. For instance, how to collect and share the evidence of hundreds of outcomes in a program implemented in 14 countries? That is where technology comes in: it helps us to process large amounts of data.”
“Oxfam Novib also was one of the first organizations to produce evidence synthesis reports, in which we draw out the strategic evidence, causal results and lessons from implementation of different programs. These reports are critical, as they allow us to make sense of complex results and trajectories. Of course we broadly share knowledge products like these. Oxfam Novib publishes and discusses a wealth of reports, evaluations, case-studies and podcasts, and provides (safe) access to evidence databases, infographics and assessments.
“I would like to mention one more thing I am proud of and which makes Oxfam Novib stand out in this field: the results that we systematize are channeled back to our partners. For example, in our iSME Development Program, we aggregate and visualize SMEs’ environmental and social impact in an online dashboard, based on the data they deliver. This dashboard guides SMEs in data-informed decision-making, while they can also use it to demonstrate their impact to potential investors. So we can all use the results of what we have mutually measured. I think that is how impact measuring should work.”
