It Stands to Reason…

Books or concrete?

Imagine you are on the board of an educational charity.  You are trying to improve the standard of education in Sub-Saharan Africa. How should you invest the charitable donations?  What would be the most effective way to spend the money?

In his book Adapt, Tim Harford tells the story of the Dutch charity, International Christelijk Steunfonds who had this problem. They funded a ‘school assistance programme’ for the Kenyan government.

Text books

The simple, most self-evident approach is to invest in books.  After all, you can’t get much education without any books to read.  

Some organisations would have made the decision, shipped the books and moved on.  Political pressure calls for fast action and it stands to reason that books would make a big difference.  Children’s lives were being wasted with every day’s delay.

But that is not what this charity did.  Instead of investing all out in books, it decided to run a trial.  It randomly allocated books to some schools whilst others received non.  A year later the charity went back and measured the difference.  It transpired that the books didn’t make any difference.  There was no discernible uplift in the performance of the schools that received the books versus the schools that didn’t.

Why was that?  One reason put forward was that the books were written in English, and English was the third language of the region.  Books aren’t much use if you can’t read them.

Visual aids

As the books weren’t useful, the charity thought that visual aids might be. They decided to try flip charts with bold graphics.   Teaching doesn’t have to be reliant on a single language.  A map of the world is a map of the world and Pythagoras’ theorem is the same in English, French, Spanish or…

So the next test the charity undertook was to provide some schools with visual aids whilst others went without.  Once again it transpired that the test didn’t make any difference at all to the standard of education.  The flip charts were a flop.

Why was that?  A suggestion put forward was that the schools all had high levels of absenteeism.  Flip charts are no good if you don’t see them.

Pills

In remote communities in Kenya, intestinal worms are rampant.  They stunt growth and cause lethargy, all of which leads to absenteeism.  So the third approach the charity tried was to provide some schools with worming tablets for their students whilst other schools went without.

This time the results were excellent.  The worming tablets reduced illness which lead to an improvement in attendance.  This in turn resulted in much better test scores.  It appears that worming tablets are far more effective at improving education than books.

Concrete

What next?  Another charity in Mexico also tried to reduce intestinal infection levels amongst children.  They tested putting simple concrete floors in people’s houses. This dramatically reduced the number of pests in houses and subsequent infections.  In Mexico concrete floors have led to a significant improvement in education amongst the poor.

Could concrete floors have the same effect in Kenya?  An ounce of prevention is worth a pound of cure. Should the Dutch charity change its investment strategy?  Or maybe just test it?

A twist in the tale

Whilst this is an interesting story about the power of trial and error, it also shows the danger of jumping to conclusions without understanding what is going on.

In the book Black Box Thinking, Matthew Syed rams this second point home.

When deciding which charity to donate to many people look at how the charity spends its money. Conventional wisdom states that one of the core metrics people look at is the proportion of money the charity spends on charitable activities.  It stands to reason that a well-run charity will minimise the amount spent on overheads and staff costs. 

The last thing a benefactor wants is his valuable donation squandered on unnecessary expenses like testing and statistics…

If somebody claims “it stands to reason” it is always worth asking “why?”

If you enjoyed this post click here to receive the next

Read another opinion

Image by James Gray-King