Chris Roche, Director at the Institute, is co-editor of the recently released book ‘The Politics of Evidence and Results in International Development. The book examines the current results agenda of international development. 

In this blog, Megan Driscoll from the Institute of Development Studies summarises a recent panel discussion on this topic.  The panel was made up of experts on aid effectiveness, including the book’s co-editor, Rosalind Eyben, and book authors, Cathy Shutt and Brendan Whitty. 

This blog was originally posted on the Institute of Development Studies website – read the original blog here.  The blog also includes a podcast of the original panel discussion.  Our thanks go to Megan for allowing us to re-post this blog.

“No one disputes the need for evidence [in Development] – but who decides what’s valid?”

This was the question Rosalind Eyben, Emeritus Fellow at IDS, posed at the start of the panel discussion, “Results and Transformational Development: What Needs to Change?” The event – the first in a series of Sussex Development Lectures – brought together four experts on aid effectiveness to examine the UK’s current results agenda of international development. Eyben was joined by Catherine Shutt and Brendan Whitty, her co-authors of The Politics of Evidence and Results in International Development, as well as Alison Evans, Chief Commissioner for the Independent Commission for Aid Impact (ICAI). Together, they presented the results agenda through a critical lens, all acknowledging its major weaknesses and clarifying where improvement must be made

Meaningless Metrics

In expanding on her chapter’s focus, Catherine Shutt remarked that DFID and other development practitioners need to make a concerted effort to move away from the use of “meaningless metrics”. This was a recurring theme throughout the night – that the constant demand to prove success through vast spreads of quantitative data is not only inefficient but also stifling to the way in which we understand progress. This seems as though it would be obvious: Development is aimed at achieving long-term, sustainable community health (isn’t it?), and surely a significant part of that health is unquantifiable, especially within a 2 or 3-year time frame. What do we do with these qualitative, intangible changes, then? Do they have any real value within the Development bureaucracy? Robert Chambers perhaps summed up this growing concern best in his question to the speakers, asking, “Can learning and changing itself be results?”

Evidence: Not All Bad?

Alison Evans had the difficult task of offering some nuanced optimism to the role of the results agenda. In doing so, Evans argued that while the agenda – or agendas, as she noted there are multiple – are often problematic in implementation, results themselves have the power to tackle deep dysfunctions in aid. She noted that the collection of systematic information allows us to understand and appraise outputs and outcomes in development in a positive manner. Moreover, she contested that transformational development and results agendas can not only coexist, but can complement one another when used properly.

Power Relations & Inconvenient Truths

The problem, Evans explained, is that practitioners examine the question of what constitutes good evidence within an extremely narrow scope. This calls back Eyben’s opening inquiry of the who – who has the power to determine which evidence is legitimate? The panel, spurred by some stimulating questions from the audience, began to unpack the power of practitioners, DFID, and policymakers when it comes to setting the agenda itself. If policymakers are the ultimate decision-makers of how aid is allocated, where does this leave us? Is it even possible if their own political agendas trump other needs?

There were two points of this discussion that I found of particular value: One was the reality of “inconvenient truths” and policymakers’ tendencies to ignore them. Evans highlighted this perfectly with an example of findings that strongly suggested that anti-corruption projects have little impact in communities, and yet continue to receive large levels of funding due to their political appeal. The other was what she referred to as the tyranny of optimism – that is, the over-asking and over-claiming of the Development business that renders practitioners all the more dependent on proof of results. Combined, these issues paint a picture of Development – and the results agenda, in particular – as a bureaucratic beast that is both overzealous and self-interested, leaving me with little faith in the fruitful role it can play that Evans had earlier described.

What about the how?

The event concluded with a series of recommendations from audience members and panellists alike for what change was needed, with ideas ranging from moving away from superfluous data, to placing greater focus on organisational behaviour and reflexive processes. Still, I was left unsatisfied by the lack of discourse on what can be done to achieve what’s needed – we talked about the who (DFID, policymakers, practitioners); we talked about the what (greater emphasis on knowledge-generation, affecting political agendas, more accountability); but what about the how? How do we make change? How do we shift policy and public priorities? How do we, as practitioners and academics, expand the scope in which evidence is examined and results are understood? This is the space I wanted to delve into, and where I felt we could have spent more time. Perhaps they are questions left for depth within Ebyen, Shutt and Whitty’s book? I suppose I’ll have to buy it and find out.

@practice4change