Just look at these outages!

What a week in technology and IT! Finally something is happening. True, nothing especially good, but at least the media have something to write about.

And, no, I am not talking about Nvidia and its ex-CEO. That’s either territory for people inclined to pass a moral judgement or for government organizations which will get involved in the national security, financial and business dealings of Nvidia.

Let’s talk about the real excitement.

Let’s talk about the Microsoft outage as reported by BBC and Reuters. The root cause? A months-old bug, failed internal connection and inability to restart servers. Rolling out a service pack for 1,094 vulnerabilities, didn’t help either.

The irony of all this is that these types of problems millions of IT personnel had to deal with before. Locked in the server rooms around the globe and trying to fix things which Microsoft created for them. Now that Microsoft is using its own software to deliver services, it is dealing with the same problems — just on a bigger scale.

Then later we had the FAA communication system outage which resulted in halting flight departures across the US. The cause? Switching part of the network from copper wire to fiber optic cable. The newly installed equipment (router) flooded the network with extra traffic, which the older equipment couldn’t handle. Further investigation should tell us if the issue was created by faulty equipment or incorrect setting.

What’s next?

The NSA internal network outage, caused by scheduled routine maintenance. I think we will never know what exactly happened there. After all it is the NSA.

And for good measure banks had issues and Apple as well.

All affected parties claimed that none of this was a result of cyber attack or AI. Not everyone shares this opinion.

The thing is that these events happened in a quick succession, but are far from unusual.

Microsoft and the updates — July, 2024 CrowdStrike distributed a security software update to Microsoft Windows machines around the world. Around the world, 8.5 million machines were disabled and had to be manually fixed. Sixty percent of Fortune 500 companies were affected. Banks, airlines, hotels, hospitals … the list is long. It took days before things went back to normal. Financial losses in billions of dollars. Over 5,000 flights were canceled and it took days to get back to normal. The cause? Human error. Here is a list of outages of Microsoft Azure cloud service. Out of 16 major events, 1 can be attributed to cyber attack. The rest to internal teams.

FAA grounding flights — January, 2023, FAA system outage (NOTAM) Airplanes grounded, flights canceled or delayed. Chaos. The cause? ‘… a contractor inadvertently deleted critical files while working to synchronize the live primary database and a backup database.’

I think that you start seeing a recurrent pattern.

Network outage — July 2022, Rogers the Canadian telecommunication provider went down for 25% of Canada. Phones, internet, payment networks — everything was gone. Government, banks, emergency services down, down and down. It took over a day to restore a majority of the service. The cause? A maintenance upgrade which caused the entire network to lose connectivity to the rest of the Internet. The funny thing was that the technicians who had to visit each affected device to manually restart it couldn’t use their cellphones to communicate. Their network was down. #chokingonirony Was it the first time? Of course not! A similar thing happened in April, 2021. The reason? A software update.

Banking system down — February 2022, major Canadian banks went offline. Online banking, mobile apps, phone systems, eTransfer, ATM all went down. The cause? An ‘internal technical issue.’

I want to emphasize that I am not making fun of or scolding people who got involved in these outages. There is always this one unlucky guy who presses the ‘Enter’ key or hits the ‘OK’ button. And I can feel their horror when they realized that something happened and that something hit the fan. I want to highlight that people are doing their best to keep everything up and running and despite their best effort, sometimes it doesn’t go according to the plan.

But here are a few things which we should be aware of, and they provide context for things to come.

First of all, technology is getting more and more complicated. Organizations are adding new systems all the time and the bigger the company the more difficult it gets. Second of all, let’s talk about ‘AI.’ I would suggest that it is better to use just the term ‘technology’ because it immediately shows the benefits and drawbacks for the IT departments. As any technology, it makes things go much faster. Faster to identify issues, and faster to exploit them. Faster to write software, faster to lose understanding what the software does. Proponents position this tech as a ‘replacement for people,’ but that will lead only to negative consequences.

Some suggested that the recent outage with Office 365 can be attributed to the announced ‘voluntary exit offer’ to 60,000 employees there. That’s nonsense. But what will happen is that, with the departure of that many people, Microsoft will lose its corporate memory and if you think that the latest outage was big, there are bigger things to come. Fortunately the marketing department at Microsoft already has a solution. First they will rename Office 365 to Office 364 and will decrease the number with every outage.

It might seem like this is all doom and gloom and like, as humans, we can’t create amazing technology. But that’s not true, and here’s a great example.

On November 18, 2026, NASA’s Voyager 1 became the first human-made object to reach a distance of one light-day from Earth — roughly 25.9 billion kilometers (16.1 billion miles) away. Cheers to human imagination and creativity.

The recurrent pattern? Despite all the setbacks, human ingenuity will always take us forward.

Next
Next

Warm freezers and hot toys. The tip of the iceberg