Does publishing performance data make people perform better?
The National Health Service in the UK has developed a cunning ruse to improve patient care.
They have started to publish data about consultant’s mortality rates. Before you go and see a specialist you can go online and see if he is competent or not. If you don’t like what you see, vote with your feet, find another doctor.
You can see the data here
“Data for better services”
Or at least that is what the strap line says, so I had a look. I chose Adult Cardiac Surgeons (A for Adult was first on the list, I’m not experiencing chest pains) and this is what I saw.
Each surgeon has been given a “risk adjusted mortality rate” and as you can see some surgeons kill more people than others — yes I know that was uncalled for, just making a point.
Put less controversially there is variation in performance.
Now I don’t know about you, but as a tax paying member of the public, and having given it great consideration, I don’t want to be seeing any quack with a score above — lets be generous — erh… 1. I have a car and I will travel. It isn’t every day that I will have a heart bypass, why on earth would I want it done by an “average” surgeon?
And the last surgeon I want is Dr. (sorry Mr.) 14 points out there all by himself on the far right.
These scores are invaluable information. Aren’t they?
Do the numbers really help?
Probably not:
- Half the scores were missing, only 214 of the 398 surgeons listed were given a score.
- According to the statistics every surgeon (including Mr. 14 points) was “within the expected range”.
- There was no indication of statistical significance, — if we both flip 10 coins we will get a different number of heads, does that make you a more competent coin flipper than me?
- Last, but by no means least there was no indication of context.
There was no indication of context?
- Perhaps Mr. 14 points performs the trickiest operations
- Perhaps Mr. 14 points works with a cack handed anaesthetist
- Perhaps Mr. 14 points catchment area is the fast food, cardiac capital of the UK
Who knows? The publishers of the data admit:
When mortality rates are higher than expected there are a number of possible explanations which include issues with the data, abnormal case mix which is not adjusted for by the risk models, issues with the hospital’s care or issues with the individual surgeon’s care. Any judgement can only be made after a full investigation of all the issues.
As a fully paid up member of the general public I have precisely no chance of making a full investigation of all the issues.
Having read this I hope you see that publishing the data — or at least the bit I looked at — is a useless ploy.
In fact, it is worse than useless, all it does is scares people.
I’m all for measuring and understanding performance
I spend most of my working life trying to minimise and reduce variation, it is a very sensible thing to do. But publishing the data only serves to confuse the public and irritate the surgeons.
I don’t know about you, but the next time I see a surgeon I’d like to see a cool, calm and collected one. Not one consumed with anger, fear and distraction. Particularly if he has a scalpel in his hand.
I’m old fashioned like that.
Drive out fear ~ W. Edwards Deming
If you enjoyed this post click here to have the next delivered straight to your inbox
Read another opinion
Image by Aleera
James,
I am in total agreement. The general population is neither educated enough, cares enough, or doesn’t have enough time to research what is behind those numbers.
What people do care about is that they will be cared for by a experienced, competent doctor who hasn’t killed anyone. Is it fair to assume that that describes all doctors? Unfortunately, no.
Perhaps the NHS had good intentions, but they fell flat. Is there a better way to convey what they are trying to?
Annette :-)
Annette, I am sure the NHS has good intentions, I’m just not so sure about their political masters
Hi James,
Interesting stats and initiative and one has to wonder about the usefulness of publicly publishing this type of data. However, I’d be more interested to find out what “within the expected range” means as that would give a whole lot more meaning and context to the data.
I’m all for monitoring and measuring performance. A really interesting case I saw recently was this: “Use of force by police officers declined 60% in first year since introduction of cameras in Rialto, Calif.” (http://online.wsj.com/articles/what-happens-when-police-officers-wear-body-cameras-1408320244)
However, this type of result probably points to much bigger, underlying issues.
Adrian
Thanks for the link Adrian, fascinating
James
One of your best posts to date. Frighteningly perceptive…
Keep them up
Thank you very much Howard
Hello James,
Thank you: it occurs to me that I am now informed on something that I knew little about.
Variation, let’s just consider variation. It occurs to me that the signature of life (as an dynamic process that patterns) is variation: diversity in species, diversity in ecologies, diversity in climate, diversity in languages, diversity in cultures, diversity in members of the same species… Variation is life; life is variation.
I invite you to consider that to set oneself the task of eliminating variation is to set oneself against life. How is the wave (a feature of life ever going to control the ocean (life itself). It is fools errand. At best one can drive variation underground and/or cause variation to show up elsewhere.
What if we celebrated variation and worked with it? You know like when you sailing a boat with sails. One does not fight the direction of the wind. One works with the wind and harnesses it by tacking – again and again to keep the attunement in place.
All the best
maz
I have a degree in Genetics Maz, variation is the source of evolution, but also the source of cancer. All things in moderation.
But a very interesting point
James
Hello James,
I invite you to consider that moderation leads to the Buddhist way of life. Through moderation one does not disturb the peace. Now I invite you to ask yourself what specifically the Buddhists have created, invented, given the world?
The way of moderation is the way of reasonableness. And as George Bernard Shaw said, reasonable people adapt themselves to the world (as it is). Therefore, all progress rests upon the shoulders of unreasonable people.
All the best
maz