"...Captain Kirk forgot to put his machine on stun..."
- reportedly commented by Voyne Ray Cox, in surprisingly good humor after his mistreatment
Therac-25 was a type of high energy cancer radiation treatment machine (Medical linear accelerator, or a linac). It was designed by Atomic Energy Commission Limited (AECL - these days called Atomic Energy of Canada Limited), Canadian crown corporation. The prototyping was finished in 1983 and eleven units total were installed in USA and Canada. The machine was quite advanced because the computer software made the system far easier to use. Basically, the operator only needed to enter the prescription on a computer terminal and the machine did the required adjustments automatically, reducing the setup time.
The Therac-25 was famous due to several faults it had, leading to 6 serious mistreatments, of which 3 were ultimately fatal, between 1985 and 1987.
The machine was controlled by a DEC PDP-11 minicomputer with a terminal. The machine could deliver two forms of treatment: electron treatment and X-Ray treatment. The electron mode would deliver a small dose directly, and the X-ray mode would deliver a full 25 million electron volt blast, filtered through a thick metal plate.
The software that run this thing had several race conditions in it, particularly in the terminal handling. In two fatal cases, the race condition was like this: The operator typed in the parameters to the machine, and cursor ended up in the command line, the machine started doing whatever magic to get the beam ready. Then the operator noted, "Oops, I said X-Ray instead of Electron," moved the cursor up to the appropriate field, corrected the mistake, went back to the command line, and waited for the thing to get ready. The machine indicated it was ready and parameters were okay, so the operator proceeded. The problem was, the metal plate was moved away, as was required by the electron mode, but the system still delivered the X-ray treatment. It turned out that if the input was given quickly enough, as is the case with routined linac operators, the system didn't know to switch from X-Ray mode to Electron mode - the machine was left in ambiguous "hybrid" state. When beam was turned on, a 25000 rad dose was delivered in about 2 seconds, and the machine just replied "Malfunction 54".
Other errors were related to feedback. The system might fail and say insightful errors like "Malfunction XX", for which no documentation was provided. The operators, of course, believed it meant that the treatment had been interrupted before it had been administered - and would try again. This lead to too high radiation doses, of course, and in case of "Malfunction 54", even a single attempt was way above the limit.
The case of Voyne Ray Cox in March 1986 combined many problems: He received three full blasts from the machine during a treatment, mostly because the terminal didn't give enough details to the operator, and the operator had no idea what was happening on the treatment table (the intercom had been broken for a long time and the video display was disconnected).
AECL tried to be rather quiet about the cases. First they denied it was even possible, and once they found the problem, they said their software fix would increase the reliability "by 5 orders of magnitude". Later, when the input race condition was found, they asked the operators to remove the "up" arrow key from the keyboards. The system was finally fixed in 1987 when they added hardware safeguards to prevent the thing from happening.
The Therac-25 case is probably one of the most famous cases of a software bug leading to loss of human lives. It was a good example of how errors can be deeply hidden in an otherwise well-working systems, how bugs can manifest themselves disastrously when combined with other mistakes.
Sources:
http://www.computingcases.org/case_materials/therac/therac_case_intro.html
http://courses.cs.vt.edu/~cs3604/lib/Therac_25/Therac_1.html
http://www.smi.stanford.edu/people/felciano/research/humanerror/humanerrortalk.html#RTFToC18
http://www.uoguelph.ca/~meby/