A Simple Mistake That Has Probably Made You Poorer

The case for austerity

The inaugural meeting of the American Economic Association was held on the 4th January 2010. Two giants in the world of economics; Professor Carmen Reinhart and former chief economist of the International Monetary Fund, Ken Rogoff were present. They presented their research paper, “Growth in a Time of Debt”.

The headline message was clear and head-turning. When the size of the country’s debt rises above 90% of Gross Domestic Product, economic growth slows dramatically.

This message, and its authors, were getting an awful lot of attention. The 90% debt-to-GDP limit was quoted by people like EU Commissioner Ollie Rehn and prominent US Republican Paul Ryan.

“Growth in a Time of Debt” strongly influenced economic policy in the UK under the coalition government.  They increased taxes and cut spending under the “age of austerity” banner.

This was heavyweight economic research being taken seriously. It was shaping the opinions of politicians around the world.

Checking the analysis

Thomas Herndon was a 28-year-old graduate student at the University of Massachusetts Amherst. He was assigned the task of replicating the results of an economics paper. He chose “Growth in a Time of Debt”, and spent a term attempting, and failing, to reproduce the results of the paper.

I remember I had a meeting with my professor, Michael Ash, where he basically said, ‘Come on, Tom, this isn’t too hard – you just gotta go sort this out’ ~ Thomas Herndon

After many failed attempts, his professor suggested that he contact Reinhart and Rogoff. They provided their original spreadsheet.

Everyone says seeing is believing, but I almost didn’t believe my eyes ~ Thomas Herndon

Herndon called his girlfriend to double-check his discovery.

A handful of problems

Herndon unearthed three issues with the analysis

1. A spreadsheet error

Reinhart and Rogoff had missed out 5 of the 20 countries in their calculation of average GDP growth of countries with high public debt. The countries they had missed were Australia, Austria, Belgium, Canada and Denmark.

2. Missing data

The data from several countries was missing altogether. There was little good quality post-war data available for Canada, Australia or New Zealand. So Reinhart and Ragoff simply left it out.

3. Simple averaging

Another problem was the way Reinhart and Rogoff averaged their data. They did not take into account the size of the country and the time the debt was held for.

New Zealand’s single year, 1951, at -8% growth is held up with the same weight as Britain’s nearly 20 years in the high public debt category at 2.5% growth ~ Michael Ash

Thomas Herndon and his professors agreed that high levels of debt correlated with lower growth. But the link is much weaker than Reinhart and Rogoff had suggested. There are also lots of exceptions to the rule.

Nobody is immune to problems with data and flaws in calculations

In this case independent checking and dogged determination uncovered the problem. However, there is a good chance that similar problems exist within your organisation. The key questions are:

• Do you have a system for uncovering existing issues?
• Do you have a way of preventing future issues like this from happening?

And the scariest question of all:

• How many decisions have you made on the basis of invalid data and interpretation?

Let’s hope that any of the existing problems you uncover haven’t crippled the global economy.

For simple tools that can help you avoid the same fate, think about using the ROKS KPI Canvas.

If you enjoyed this post click here to receive the next

Read another opinion

Image by Ivan Rigamonti