It’s been quite a few years now that I have been teaching hundreds of students on the limitations of governance indicators, especially the ‘big data’ ones. As someone who was educated first as a scientist and only later as a social scientist, I have never quite felt comfortable with data that I did not collect myself. It seems like a day doesn’t go by without an NGO announcing a new index or a corporate consultancy selling some miracle forecast. Yet primary data collection in the field of governance is extremely scarce: because of problems with conceptualization, the overly ambitious goal of comparing countries by the hundreds rather than the tenths, and, therefore, the inability to meet the real costs of quality data production. Consequently, as governance scholars, we work mostly with the equivalent of ‘derivatives’ in finance. Somebody packed some smaller packages which in their turn, like Russian dolls, contain some others that nobody knows where they come from and what exactly they contain. These are then standardized, normalized and messed with until they deliver what people want. Refined calculations try to separate the statistical noise from the content and find noise often stronger. Moreover, index authors omit the fact that the content itself is not based on facts, but rather on human judgement influenced by ideology or severely constrained by a limited opportunity to make direct observations.
In this sad landscape Doing Business was somewhat of an exception. Many of its elements – like how many days it takes to get electricity installed when you open a company – were entirely fact-based, unambiguous, and actionable. Doing Business served to incentivize reforms of the best kind – to remove the barriers that gatekeepers create to collect supplementary informal taxes on firms and bribes to themselves. This is precisely why Doing Business was created. It offered an entire generation clear guideposts for emulating successful economies.
For our purposes as corruption scholars, Doing Business was also a valuable instrument, together with fiscal transparency and a few other fact-based tools which stood far above a mass of indexes made up of derivatives. For instance, the time it takes to pay taxes or to import and export goods influences the degree of corruption. This is why we included them in our composite Index for Public Integrity (IPI) among the determinants of corruption opportunities. It sounds intuitive that governments that do not make people queue to pay taxes or grant import-export permits selectively do better on both integrity and performance. We published evidence to this effect. Such evidence would not have been possible had not the World Bank collected the data in the first place.
In this context, the Wilmer Hale auditors’ report – which found that the top management of the World Bank pressured the research team of Doing Business to change post factum the methodology and manipulate rankings to make some big funders (or potential big funders) of the likes of China or Saudi Arabia look better – is a severe blow to the credibility of the indicators. It is also a blow to the hope that we can have transparent and fact-based indicators in general. If one can manipulate such simple things (e.g. adding Hong Kong data to that of mainland China), what is the difference with the non-transparent corruption risk estimates? Why invest in data collection if the treatment of data cannot be trusted? And if the World Bank does not have the financial resources to fund such an exercise and guarantee its impartiality, who does? No similar data collection exists, although a variety of foundations, each according to its own ideology, produce indicators on the quality of bureaucracy or freedom from regulation.
Anticorruption is a political endeavour and having the World Bank or any other intergovernmental organization where the board is formed by the countries it is supposed to monitor engage in this exercise is bound to be problematic. So far, the World Bank had shown itself through the excellence of its research, if not by the efficiency of its anticorruption programs. But now it has become problematic not just for the Bank, but for everybody using these scores: academics, reformers, journalists. We are not able to trust these figures anymore and yet we are unable to replace them – at least not on such a scale. The evidence of gaming and manipulation is older than this scandal, and the countries which required political intervention do surprisingly well in other rankings, too.
And yet we should not discard fact-based indicators and return to derivatives just because of this incident. We should see the value of the fact that this is a public audit and a public confession. We should encourage the World Bank and the International Monetary Fund to rid themselves of the managers involved and to reform their research so that political intervention is no longer possible, rather than abandoning it. This means including external stakeholders in some permanent oversight capacity, from design to final reporting – academics, think-tanks, and experts from civil society. Had such scores been produced in partnership with big business associations, or civil society coalitions, then the intervention could not have taken place. Strong countries which perform poorly are always tempted to intervene. When I launched the first IPI at OECD the first country who dispatched an emissary was China – to politely question why instead of Facebook users (where China does badly, since it is forbidden) we cannot use their government-controlled channel. I told them equally politely that we include the number of people associated through social media in our Enlightened Citizens indicator as a proxy for the citizenry capacity to rally against the government if needed– so I could hardly follow their suggestion. I think we parted in mutual respect- but then I have never sought financing from China.
Killing Doing Business altogether, while allowing all the derivative indexes to carry on (by the way, many included Doing Business, but as nobody knows what’s inside they will just let it drop as if nothing had been) strikes me as a step back. It is true that nearly the entire team was compromised insofar as they had to comply and did not blow the whistle in a timely manner. No doubt the entire governance research will need to be reorganized, under a different culture and perhaps external leadership, not just an external board. The practice of having research teams led by external academics is not new in international organizations and it has always worked well. Clearly, it makes political intervention more remote.
While we all wait to see what the Bank will do next (and IMF, where one of the main managers incriminated, Kristalina Georgieva, has a leading position) we also need to do something to continue our work, while keeping the trust of the many who use the actionable indicators that we create.
Already this week, the IPI will remove the countries suspected or proven of political intervention altogether, especially since suspicion exists this is not the only rating where foul play existed. Starting next Monday we shall remove China, Saudi Arabia, the United Arab Emirates, and Azerbaijan from the IPI. We shall keep them in our De facto Transparency Index (T-Index), where the web-based evidence is collected directly by us and verifiable by every reader, who also has a feedback form to report problems.
Furthermore, for the future editions of the IPI, we shall use the T-index to replace the parts which relied on Doing Business. This is both a gain and a loss; a gain, since T-index is the first direct measure of computer-mediated transparency for over 100 countries, and a loss, since we have nothing to replace the time one pays taxes or exports, which we know are significant and need improvement in so many countries.
We need to remind ourselves and others that trust is the most valuable currency: and we should all do our bit to preserve or regain it in order to continue to offer real and accurate maps for those who navigate the difficult waters of governance reform.
—
Alina Mungiu-Pippidi is a Professor of Democracy Studies at Hertie School in Berlin. Together with her research teams, she created the data repositories and fact-based governance indicators on www.europam.eu , www.opentender.eu, and www.againstcorruption.eu, aside from the Index for Public Integrity.
