Preserving the integrity of corruption measurement

You may say it’s a joke, but it’s not. Why, in recent years, do we have such a congestion of corruption measurements, especially among international organizations? For many years, only the World Bank and Transparency International did that and led the field by similar, limited but transparent methods: aggregating all expert scores or organizing public opinion surveys of citizens or firms. Then victimization surveys proliferated—some, like the Mexican one, becoming established—asking people if they were coerced to bribe. In all this interval, we missed objective evidence, but all these perception indicators, which received tons of criticism—some justified, some not—had managed to map the world.

The academia was not idle in this interval. In the last fifteen years, organizations like ERCAS and the Government Transparency Institute, supported by EU and US research funding, produced another generation of objective indicators. These measurements are based on directly observable indicators such as spending, public procurement, digital transparency, and regulation—in the case of the Index for Public Integrity, a full model of enablers and disablers of corruption. These new indicators allow for analytical insights as well as actionability. In the T-index, improvements in government transparency can be observed in real-time, with crowd-sourcing ensuring that no fixing or gaming can take place. Unfortunately, the excellent Doing Business World Bank set of indicators met its untimely end due to undue influence from countries seeking to improve their reputation, such as Azerbaijan, Saudi Arabia, UAE, and China, to name the most notorious.

You would say that this is a positive trend, but of course when reputation by governance is concerned, it never really is. China, Russia, Azerbaijan, and Saudi Arabia have been the top funders of the internationally endorsed Anticorruption Academy in Vienna (IACA) which has struggled in the last years to create a new measurement of corruption. The ball and the Saudi funding have now passed to the UNDP, an organization with no experience in the field.

Yesterday I had to answer at IACC in Vilnius (in a panel with the World Bank, UNODC, and Transparency International) a question from the Saudi Arabia representative. Why am I an opponent of corruption measurements sponsored by their government? Why did I decline the repeated invitations from this country, and others, to join in any capacity and advise instead of criticizing and boycotting their effort?

 

18 June, 2024, The International Anti-Corruption Conference, Vilnius

The answer is very simple. Only an organization with no interest can do corruption rankings, and one that is academically fit and economically autonomous, not depending on the funds of patrimonial countries seeking to improve their non-improvable reputations, especially when it happens that their internal critics may be discretionarily arrested, kidnapped, or eventually killed. The world has enough academics, able to win transparently enough research funds—both ERCAS and GTI just won, with their partners, 6-million-euro research grants which do not depend on the favor of any government, just on anonymous academic reviewers.

Why don’t you leave academics do their rankings without funding them? Surely it would be more rewarding than the elusive quest of a measurement to make you look better? No need for gifts to get objective treatment from us!  Looking better is really up to you. You do the reforms, and we shall provide the mirror. If they are the right reforms, and they have an impact, you will discover that your reputation will improve eventually.

The Doing Business Gate: Let’s not throw the baby out with the bathwater

It’s been quite a few years now that I have been teaching hundreds of students on the limitations of governance indicators, especially the ‘big data’ ones. As someone who was educated first as a scientist and only later as a social scientist, I have never quite felt comfortable with data that I did not collect myself. It seems like a day doesn’t go by without an NGO announcing a new index or a corporate consultancy selling some miracle forecast. Yet primary data collection in the field of governance is extremely scarce: because of problems with conceptualization, the overly ambitious goal of comparing countries by the hundreds rather than the tenths, and, therefore, the inability to meet the real costs of quality data production. Consequently, as governance scholars, we work mostly with the equivalent of ‘derivatives’ in finance. Somebody packed some smaller packages which in their turn, like Russian dolls, contain some others that nobody knows where they come from and what exactly they contain. These are then standardized, normalized and messed with until they deliver what people want. Refined calculations try to separate the statistical noise from the content and find noise often stronger. Moreover, index authors omit the fact that the content itself is not based on facts, but rather on human judgement influenced by ideology or severely constrained by a limited opportunity to make direct observations.

In this sad landscape Doing Business was somewhat of an exception. Many of its elements – like how many days it takes to get electricity installed when you open a company – were entirely fact-based, unambiguous, and actionable. Doing Business served to incentivize reforms of the best kind – to remove the barriers that gatekeepers create to collect supplementary informal taxes on firms and bribes to themselves. This is precisely why Doing Business was created.  It offered an entire generation clear guideposts for emulating successful economies.

For our purposes as corruption scholars, Doing Business was also a valuable instrument, together with fiscal transparency and a few other fact-based tools which stood far above a mass of indexes made up of derivatives. For instance, the time it takes to pay taxes or to import and export goods influences the degree of corruption. This is why we included them in our composite Index for Public Integrity (IPI) among the determinants of corruption opportunities. It sounds intuitive that governments that do not make people queue to pay taxes or grant import-export permits selectively do better on both integrity and performance. We published evidence to this effect. Such evidence would not have been possible had not the World Bank collected the data in the first place.

In this context, the Wilmer Hale auditors’ report – which found that the top management of the World Bank pressured the research team of Doing Business to change post factum the methodology and manipulate rankings to make some big funders (or potential big funders) of the likes of China or Saudi Arabia look better – is a severe blow to the credibility of the indicators. It is also a blow to the hope that we can have transparent and fact-based indicators in general. If one can manipulate such simple things (e.g. adding Hong Kong data to that of mainland China), what is the difference with the non-transparent corruption risk estimates? Why invest in data collection if the treatment of data cannot be trusted? And if the World Bank does not have the financial resources to fund such an exercise and guarantee its impartiality, who does? No similar data collection exists, although a variety of foundations, each according to its own ideology, produce indicators on the quality of bureaucracy or freedom from regulation.

Anticorruption is a political endeavour and having the World Bank or any other intergovernmental organization where the board is formed by the countries it is supposed to monitor engage in this exercise is bound to be problematic. So far, the World Bank had shown itself through the excellence of its research, if not by the efficiency of its anticorruption programs. But now it has become problematic not just for the Bank, but for everybody using these scores: academics, reformers, journalists. We are not able to trust these figures anymore and yet we are unable to replace them – at least not on such a scale. The evidence of gaming and manipulation is older than this scandal, and the countries which required political intervention do surprisingly well in other rankings, too.

And yet we should not discard fact-based indicators and return to derivatives just because of this incident. We should see the value of the fact that this is a public audit and a public confession. We should encourage the World Bank and the International Monetary Fund to rid themselves of the managers involved and to reform their research so that political intervention is no longer possible, rather than abandoning it. This means including external stakeholders in some permanent oversight capacity, from design to final reporting – academics, think-tanks, and experts from civil society. Had such scores been produced in partnership with big business associations, or civil society coalitions, then the intervention could not have taken place. Strong countries which perform poorly are always tempted to intervene. When I launched the first IPI at OECD the first country who dispatched an emissary was China – to politely question why instead of Facebook users (where China does badly, since it is forbidden) we cannot use their government-controlled channel. I told them equally politely that we include the number of people associated through social media in our Enlightened Citizens indicator as a proxy for the citizenry capacity to rally against the government if needed– so I could hardly follow their suggestion. I think we parted in mutual respect- but then I have never sought financing from China.

Killing Doing Business altogether, while allowing all the derivative indexes to carry on (by the way, many included Doing Business, but as nobody knows what’s inside they will just let it drop as if nothing had been) strikes me as a step back. It is true that nearly the entire team was compromised insofar as they had to comply and did not blow the whistle in a timely manner. No doubt the entire governance research will need to be reorganized, under a different culture and perhaps external leadership, not just an external board. The practice of having research teams led by external academics is not new in international organizations and it has always worked well. Clearly, it makes political intervention more remote.

While we all wait to see what the Bank will do next (and IMF, where one of the main managers incriminated, Kristalina Georgieva, has a leading position) we also need to do something to continue our work, while keeping the trust of the many who use the actionable indicators that we create.

Already this week, the IPI will remove the countries suspected or proven of political intervention altogether, especially since suspicion exists this is not the only rating where foul play existed. Starting next Monday we shall remove China, Saudi Arabia, the United Arab Emirates, and Azerbaijan from the IPI. We shall keep them in our De facto Transparency Index (T-Index), where the web-based evidence is collected directly by us and verifiable by every reader, who also has a feedback form to report problems.

Furthermore, for the future editions of the IPI, we shall use the T-index to replace the parts which relied on Doing Business. This is both a gain and a loss; a gain, since T-index is the first direct measure of computer-mediated transparency for over 100 countries, and a loss, since we have nothing to replace the time one pays taxes or exports, which we know are significant and need improvement in so many countries.

We need to remind ourselves and others that trust is the most valuable currency: and we should all do our bit to preserve or regain it in order to continue to offer real and accurate maps for those who navigate the difficult waters of governance reform.

Alina Mungiu-Pippidi is a Professor of Democracy Studies at Hertie School in Berlin. Together with her research teams, she created the data repositories and fact-based governance indicators on www.europam.eu , www.opentender.eu,  and www.againstcorruption.eu, aside from the Index for Public Integrity.

 

Saudi Arabia

Saudi Arabia has made some progress on e-government and administrative simplification over the past decade, being credited by Global Competitiveness Report 2019 with a judiciary at the level of US and UK. However, this alleged progress is offset by its performance on e-citizenship, budget transparency, trade openness and freedom of the press, where it is on the bottom of its income group. There is considerable room to progress without major political change simply by pushing more decisively the agenda of e-government on the model of United Arab Emirates.

Social Media Provides for Increased Criticism of Corruption in Saudi Arabia

Social media has been increasingly used as a vehicle for raising awareness about corruption and other abuses in Saudi Arabia. Platforms such as Youtube and Twitter have become a means of spreading comments and satirical videos on episodes of corruption or undue political influence, topics that have always been considered taboo in the country. This has given a new dimension to freedom of expression in a country where conventional media still faces severe restrictions.

Some demographic aspects of Saudi Arabia are favorable for this new dynamic. The population is very young, with about 70 per cent under 30 years of age, and internet penetration is around 40 per cent. This makes a large pool of potential followers of the online comedy shows, bloggers and twitter users that use the internet to express their criticism of government officials and even members of the royal family.

One of the latest online hits in the country is a video mocking the Ministry of Commerce for its ambigious enforcement of business regulations, which usually favors powerful businessmen. An anonymous writer on Twitter has also become popular with posts about misconduct by members of the royal family.

The government has already felt the impact of this intensification of the online political debate. Legislation that extends restrictions to traditional media also to blogs and social media platforms has been introduced, and analysts believe that government monitoring of online media will be strengthened in the near future.

For more details read the article “Social media skewer corruption in Saudi Arabia” on vancouversun.com. The picture shown above is featured in the article and is credited to Susan Baaghil/Reuters.