The AT&T Network Crash of 1990
Intro
On January, 15 1990 more than half of the AT&T network crashed. More than 75 million were unanswered during the 9 hours as AT&T attempted to fix and find the problem.
What Happened?
Despite the beliefs of law enforcement and telco security that hackers were responsible for the crash, the real problem was an embarrasment for AT&T during a time of growing competition.
The cause was a bug in a routine software update of it 4ESS systems in December. Something happened to the particular switch (Switch A) that caused the disaster and it sent out a broadcast declaring it was no longer accepting any new messages. Once Switch A was back up, it sent out another broadcast telling the other switches that it's back online. That's how it's supposed to work. However, the bug was, while one of the switches (Switch B) was still resetting itself to acknowledge Switch A being back in service, Switch A sent this one another message, confusing it. Switch B went down with the same problem, and it resulted in a chain reaction. As the network continued to repeat this problem.
The Solution & Fallout
AT&T solved the problem by reducing the messaging load of the network. That allowed the switches to rest themselves and the network to stabilize. Despite AT&T's press release for the crash, it gave a much stronger need to execute Operation Sun Devil, the nationwide crackdown on hackers in the early 1990s.