WELLINGTON – A faulty software update caused technological havoc worldwide on Friday, grounding flights, knocking down some financial companies and news outlets, and disrupting hospitals, small businesses and government offices.
The breadth of the outages highlighted the fragility of a digitized world dependent on just a few providers for key computing services.
The trouble was sparked by an update issued by cybersecurity firm CrowdStrike and only affected its customers running Microsoft Windows, the world’s most popular operating system for personal computers. It was not the result of hacking or a cyberattack, according to CrowdStrike, which apologized and said a fix was on the way.
Businesses and governments worldwide experienced hourslong disruptions — their computer monitors glowing blue with error messages — and they scrambled to deal with the fallout. CrowdStrike’s CEO said some of their systems will require time-consuming manual fixes.
Thousands of flights were canceled and tens of thousands were delayed, leading to long lines at airports in the U.S., Europe, Asia and Latin America. Airlines lost access to check-in and booking services in the heart of the summer travel season. By late afternoon Eastern time, the worst appeared to be over, though there were still lingering cancellations and delays do to the cascading effect of the disruption.
Several local TV stations in the U.S. were prevented from airing the news early Friday, and some state and local governments reported problems at courts, motor vehicles departments, unemployment agencies, emergency call centers and other offices, but as the day progressed many of the systems were getting back to normal.
Affected hospitals had problems with appointment systems, forcing them to suspend patient visits and cancel some surgeries.
Alison Baulos said her 73-year-old father’s heart surgery in Paducah, Kentucky, was canceled Friday morning because of the tech outage, leaving her family scared and worried.
“It does really make you just realize how much we rely on technology and how scary it is,” Baulos said in an interview. She said her father was waiting at Baptist Hospital to find out what will happen next. A phone message left with the hospital was not immediately returned.
American Express said it temporarily had some difficulties processing transactions, while TD Bank responded to online complaints by saying it was working to restore customers’ ability to access their accounts.
Elsewhere, people experienced minor inconveniences, including trouble ordering ahead at Starbucks, causing long lines to form at some of the coffee chain’s stores.
In New York City’s Times Square, right before 12:30 a.m., the blue “recovery” screens popping up on laptops appeared on several giant electronic billboards. A few were dark Friday afternoon.
A disturbing reminder of vulnerability
Cyber expert James Bore said real harm would be caused. “All of these systems are running the same software,” Bore said. “We’ve made all of these tools so widespread that when things inevitably go wrong — and they will, as we’ve seen — they go wrong at a huge scale.”
The head of Germany’s IT security agency, Claudia Plattner, said “we can’t expect a very quick solution.” A forecast for when exactly all systems will be up and running is difficult, but “it won’t be hours,” she added.
CrowdStrike said in a recording on its customer service line that the problem was related to “the Falcon sensor,” referring to one of its products used to block online attacks. The company says it has 29,000 customers.
In an interview on NBC’s “Today Show,” CrowdStrike CEO George Kurtz apologized, saying the company was “deeply sorry for the impact that we’ve caused to customers, to travelers, to anyone affected by this, including our companies.”
“We know what the issue is” and are working to fix it, Kurtz said. However, he noted it could take “some time” for certain customers, especially those lacking in-house expertise.
While CrowdStrike’s update was automated, the fix requires hands-on work such as deleting corrupted files, which could take some customers days or longer, said Forrester analyst Allie Mellen.
“Given that CrowdStrike has a ton of customers, a ton of Fortune 500 customers, and they have likely millions of (computers) under management, this causes a bigger issue,” Mellen said. “It is going to be a long and arduous process.”
Ann Johnson, corporate vice president and deputy chief information security officer at Microsoft, said late Friday afternoon that “at this point in time, I would say that customers are receiving or have received the necessary information they need and are getting the support they need — understanding it’s a very major issue.”
She said Microsoft’s primary focus is getting customers back online, but could not estimate how long this might take.
In Alaska, the state’s court system returned to functionality after repairs that took 12 hours to complete, according to spokesperson Rebecca Koford. In Iowa, Gov. Kim Reynolds said the state’s critical technology systems were up and running again by mid-afternoon.
Shares of CrowdStrike, which is based in Austin, Texas, ended Friday trading down more than 11%. Microsoft’s stock price fell less than 1%.
Though the outage’s impact could be felt far and wide, the forecasting firm Capital Economics said it was likely to have little impact on the world economy.
Cybersecurity experts said those affected by the outage also needed to be wary of bad actors reaching out claiming they can help. “Attackers will definitely prey on organizations as a result of this,” said Gartner analyst Eric Grenier.
In a letter to customers posted on CrowdStrike’s website, Kurtz said the outage did not affect its Falcon systems or its security scanning.
Air travel delayed everywhere
Most airlines attributed the problems to their booking systems. Thousands of flights were affected in the U.S. alone, though by late morning on the East Coast airlines said they were beginning to mitigate problems and resume some service. Unclogging the system takes time, though.
At Minneapolis-Saint Paul International Airport, Sarah Schafer was delayed getting to her cousin’s 50th birthday party in Florida. She had been waiting for almost three hours with no indication of when her flight would be rebooked.
“I seem calm,” said Schafer, who was using a cane because of ankle injury. “But my angry side might come out.”
Airlines and railways in the U.K. experienced long wait times. And airports across Europe suspended landings or halted takeoffs for several hours due to difficulties in checking in passengers.
Saskia Oettinghaus, a member of the German Olympic diving team, was among those stuck at the Berlin Airport.
“We are on our way to Paris for the Olympic Games and now we are at a standstill here for the time being,” Oettinghaus said.
In Cancun, Mexico, the main tourist destination in the Caribbean coast, the state government said there were 24 cancellations and 100 delayed flights. Some travelers tried to liven up the long waits by singing the traditional Mexican song, “Cielito Lindo,” while a band that was also stranded was playing in the Cancun airport.
Broadcasters go dark, surgeries delayed, ‘blue screens of death’
In Australia, national news outlets — including ABC and Sky News Australia — were unable to broadcast for hours. Some news anchors went on-air from dark offices, in front of computers showing blue error screens.
In the U.S., KSHB-TV in Kansas City, Missouri, aired Scripps News instead of local news until about 5:35 a.m., the stations said on its website. Other local stations owned by Scripps reported similar problems, though Scripps spokesman Michael Perry said early Friday that 90% of stations were able to air local news.
Hospitals in different countries also reported problems.
Britain’s National Health Service said the outage caused problems at most doctors’ offices because appointment and patient-record systems were affected.
At Mass General Brigham, the largest health care system in Massachusetts, all scheduled non-urgent surgeries, procedures, and medical visits were canceled Friday because of the outage, according to a spokesperson.
Some international shipping was disrupted, too.
A major container hub in the Baltic port of Gdansk, Poland, said it was battling problems. And at the twin ports of Los Angeles and Long Beach, marine terminals were affected, although the outage didn’t cause significant disruptions.
___
This version corrects the spelling of the last name of the spokesperson of the Alaska Court System to Koford.
___
Kurtenbach reported from Bangkok, McHugh from Frankfurt, Graham-McLay from Wellington, New Zealand, Hadero from New York and Ortutay from Oakland, California. Associated Press journalists around the world contributed.