Information Overload

A broken valve

In the early hours of the morning on March 28^th 1979, a valve broke on a water pipe. That pipe was at the nuclear power plant on Three Mile Island in Pennsylvania.  The failure resulted in the most serious nuclear incident that the USA has ever seen.

Chain reaction

The broken valve led to an alarming sequence of events.  I’ve listed them out below as best I can.  I’m not a nuclear engineer and have no doubt misrepresented the facts in my attempt to explain.  But I urge you to read it — there will be a test…

1. Two days before the incident, maintenance workers left a valve shut.
2. The evening before the incident another valve was damaged during cleaning.
3. At four in the morning that valve broke and a cooling pump stopped.
4. The nuclear reactor started to heat up.
5. As it got hotter, the pressure of the water in the reactor’s cooling system started to rise.
6. A pressure relief valve opened, venting contaminated water into a containment tank.
7. Three pumps started up, providing cooling water to the overheating system.
8. Unfortunately the valves on the emergency cooling pipes were closed (point 1).
9. The warning lights started to flash. 
10. One light hidden behind a maintenance tag, the other was tucked away out of sight.
11. The warning lights went unnoticed.
12. As cooling water wasn’t reaching the reactor, the temperature continued to rise.
13. As the temperature rose, control rods lowered into the reactor.
14. The nuclear reaction stopped.
15. The latent heat of the radioactive material continued to heat the water.
16. Contaminated water continued to flood out of the pressure relief valve.
17. Pressure dropped back to normal.
18. A motor kicked in to close the pressure relief valve.
19. The pressure relief valve stuck open.
20. An indicator light lit up to show the motor to close the valve was running.
21. Operators mistook this to mean the valve was closed and the system was sealed.
22. Contaminated cooling water continued to escape via the open pressure relief valve.
23. As coolant flooded out of the system, the pressure continued to drop.
24. A second set of emergency cooling pumps started up.
25. 1,000 gallons of water per minute started to enter the reactor cooling system. 
26. Not realising the pressure valve was open, operators worried that the reactor would flood.
27. They shut down the second set of emergency cooling pumps.
28. After eleven minutes contaminated water started to spill out of the containment tank.
29. In the first 100 minutes of the accident almost 32,000 gallons of contaminated water escaped.
30. After two hours, the cooling water in the reactor dropped below the level of the nuclear core.
31. The core started to melt and produce radioactive hydrogen gas.
32. An operative from a new shift realised that the pressure valve was stuck open.
33. They managed to shut it.  Cooling water stopped leaving the reactor.
34. By this time the nuclear core was in melt down and continued to heat up.
35. In the early afternoon the build up of hydrogen caused an explosion.
36. Radioactive hydrogen escaped from the building.
37. Several hours later the partially melted core was brought down to a controllable temperature.
38. The emergency ended.

I have over simplified that horribly, maybe even missed a critical fact or two, but I hope you get the impression that:

• All that went wrong was couple of faulty valves and a misreading of some indicator lights.
• The system was complicated.  
• One problem resulted in another, and another.
• When things went wrong, they started to go wrong very quickly.
• The operators didn’t know what was happening.

The aftermath

39,950 people lived within five miles of Three Mile Island. They were lucky.  The release of radioactive material into the environment was only minor.  They each received a dose of radiation that was less than a chest x-ray.  

But, as you can imagine, passions ran high and there were cries to find those responsible. A full-blown investigation came hot on the accident’s heels.

The test

I did promise one…

If you were the investigator what would you think was the real cause of the accident?  Would you plump for:

1. Operator error
2. Lack of training
3. Ergonomics
4. Poor management
5. Faulty equipment

What do you think?

The investigation

The investigators didn’t pull their punches.  You can read the full report here.  Here are some of their rather more damming observations (it reads like a freight train):

• The control panels displayed over 1,900 different pieces of information.
• Of these, anybody below about 5’5″ (1 in 20 men) couldn’t see 503 of them when standing directly infront of the panels.
• The information needed by the plant operators was often poorly located, ambiguous, or difficult to read. Bizarrely — given the number of dials and lights — it was also often non-existent.
• The alarms were poorly organised, not colour coded and they were not arranged in priority order.
• Labelling of controls and displays was inadequate or ambiguous.
• There was little consistency in the nomenclature used in procedures and used on the panel.
• The dials and indicators rarely showed what the “right” or “wrong” reading was. 
• Procedures placed too much reliance on operator short-term memory.
• Operators were burdened with unnecessary information.

On a positive note the inspectors declared that the training program was in full compliance with government-imposed standards.  Unfortunately it also concluded that these standards were inadequate.

The answer to the test is…

I think we could argue a case for all 5 options, but my favourite is ergonomics. The control panel was simply badly designed, nobody could interpret what it was saying.

Imagine you were an operator in the control room that morning.  Over 100 hundred alarms went off in the control room during the first few minutes of the accident.  If you had had any imagination at all, you would have been scared to death.  Is it any wonder that there was “operator error”?

Whether or not operator error explains this particular case … we are convinced that an accident like Three Mile Island was eventually inevitable.

I guess you don’t work in a nuclear power station.

But if you work in a hospital, or a bank or maybe an IT service centre your working environment is complicated. An error will cause all sorts lots of unseen ramifications and has the potential to go quite horrifically wrong.

As for your “control panel”, I will take a fair sized bet it is set of PowerPoint slides or spread sheets with far more information crammed onto them than you can hope to understand.

What could possibly go wrong?

More things should not be used than is necessary

William of Occam

If you enjoyed this post click here to receive the next

Read another opinion

Image by Curtis Bathurst