CrowdStrike outage aftershocks cause Delta to cancel over 1,000 more flights https://engt.co/3LTBwWN
Engadget’s Post
More Relevant Posts
-
Network Engineering | IT Operations | Leadership Development | Project Management | KPI Implementation
It's a trap. IT teams across the globe are breathing a little bit easier this morning in the wake of Delta's continuing struggle to recover from the CrowdStrike incident. Not that anyone enjoys seeing other professionals under an intense spotlight, but it is tempting to say, "We recovered far faster than Delta did!" As technology leaders, we've all been there. Something breaks, typically at an inopportune time, interrupting service to significant portions of our customers. What happens after service is restored is more important than the work of restoration. Because whatever task was done to fix the outage does not fix the underlying problem. Consider the post-mortem exchange from a memorable outage at one of my previous engagements (distilled for brevity - this exchange transpired over a couple of days): CA: The network report shows a big outage over the weekend. What happened? - That wasn't an outage. We lost a link between Data Center A and Data Center B, but customer traffic rerouted. CA: Why does the report show that multiple nodes were offline? - The monitoring platform lives in Data Center A. When the link went down, the monitoring service could not reach Data Center B. The nodes connected to B showed offline in monitoring but were actually online via the link to Data Center C, so customers weren't affected. CA: Tech Support reports show increased calls from that period, so I looked at the activity graphs for Node RandomName. Why do the graphs show a dramatic drop in customer activity at that node? - That node is using IP Addresses that are advertised from Data Center A, but it is connected to Data Center B. When the link went down, Internet traffic could not reach those customers. That node actually did go offline. CA: Do we have other nodes with similar configurations? - Yes. CA: When can we fix them? Two points to consider: 1 - Ask questions. Even though we are tempted to lead by giving direction, we rarely have the complete picture. Keep asking questions until your team is on the right track. 2 - Ask the right kind of questions. How, What, When. Never Who, and rarely Why. In other words, find the breakdown in the internal process before looking for someone to blame. If you fire the person who made a mistake, what will keep the next person from making the same error? The temptation to "fire" a vendor in the case of CrowdStrike is the trap. It still leaves the underlying problems: How well can we recover from an error? Is the recovery process well-defined? How can we mitigate errors from other vendors? #CrowdStrike #NetworkEngineering #Troubleshooting
Delta faces federal investigation as it scraps hundreds of flights for fifth straight day
cbsnews.com
To view or add a comment, sign in
-
The bill for downed IT systems can quickly rack up — especially if outages go unresolved for days. Delta Air Lines learned that lesson in July, after a faulty CrowdStrike update sent 40,000 of its Microsoft-powered computers offline. Recovery was especially tricky given the airlines heavy reliance on Microsoft and CrowdStrike, CEO Ed Bastian said in an interview with CNBC. The outage resulted in a $500 million loss for the business, including missed revenues and unforeseen compensation costs. Now, Delta is preparing to sue the providers in hopes of recouping at least some of the sunk costs. Here's more on the airline's pricey IT woes and next steps: https://lnkd.in/eafFTp-G
Delta grapples with $500M in CrowdStrike outage costs
ciodive.com
To view or add a comment, sign in
-
Family Guy | Inventor | Founder of Constant Mountain | Driving Product Innovation at Zebra Technologies
I may regret this post in the future. I hope I don't. It's tempting to blame others. Don't do it. Own it. Like a many others, I was Delta-ed this week. Whatever - this post isn't about that. It's about owning up to mistakes. Here's the opening sentence of Delta Air Lines' statement. "Like many companies worldwide, Delta Air Lines was impacted on Friday morning by an outside vendor technology issue, which prompted us to pause flying while our systems were offline." The statement immediately tries to frame Delta as a victim. They are not. They blamed their vendor - and they didn't even blame the right one. Further down it reads: "Specifically, the issue impacted the Microsoft Windows ..." Follow the link below if you want to read it all. The issue was with CrowdStrike, NOT Microsoft. The general public has never heard of Crowdstrike. Delta name dropped a company that everyone knew. What they didn't say was that this was preventable if they had adequate IT Resiliency and Quality Assurance Processes in place. I feel really bad for the stranded travelers. I appreciate the good people working at Delta who are trying to help. They've been put into a bad spot by their leaders and they deserve better. But - back to the main point. When you make a mistake, Own it. Please let me know if you agree - even more importantly - please call me out if I'm wrong. BONUS ROUND : For fun, name the specific location in this picture! LinkedIn article link: https://lnkd.in/ejYexC63
To view or add a comment, sign in
-
-
🛫 In the aviation industry, a single security giant's oversight can spell catastrophic consequences💥. Look what Delta got from the CrowdStrike and Microsoft Outage? 💰 Direct Economic Impact: 7,000 Flight Cancellations 💔 Irrevocable Trust Loss: Over 3,000 Customer Complaints 🗯 Reputation Damage: Public Criticism by State Authorities But this nightmare could have been easily mitigated with a simple, user-friendly, and efficient disaster recovery tool like HyperBDR💙! More and more airlines are choosing to partner with us to safeguard their operations🤝. Don't let the past disruptions dictate your future. 🔗 Connect with us at enquiry@oneprocloud.com to learn how HyperBDR can be your lifeline in protecting your business against unforeseen IT outages. #CrowdStrike #Delta #MicrosoftOutage #airlines #aviationindustry #HyperBDR #OneProCloud #CloudDisasterRecovery #OperationalContinuity #OperationalResilience #Sustainability #WorkflowAutomation #DigitalTransformation #DataProtection
To view or add a comment, sign in
-
-
I.T. Professional, Senior System Administrator, Healthcare Analyst, Teacher, Calligrapher and a person who enjoys running.
Delta Airlines aftermath from CrowdStrike's mistake My opinion... Delta Airlines is playing the blame game. Blaming CrowdStrike and Microsoft for the three day meltdown of their systems. Yes, CrowdStrike is at fault for updating the content file in the drivers directory of Microsoft kernel that caused BSOD. I don't know what course of action, assistance (if any) and the infrastructure that Delta Airlines deployed. I do know that American and United Airlines were up and running in one day. As an I.T. professional working the front line, it's hard to imagine the I.T. team at Delta is not working their best to bring up servers and workstations. It is the CIO and CTO of Delta who set the RPO and RTO and provide redundant systems, resources, methodology and processes to recover from a disaster. As a casual observer, 3 days vs. 1 day, Delta has a lot work in front of them.
To view or add a comment, sign in
-
Founder and CISO <𝐒𝐭𝐞𝐞𝐩𝐥𝐞 𝐂𝐲𝐛𝐞𝐫> | Healthcare Technology & Cyber Specialist | Mentor | UK Cyber Security Council Chartered Status Member
One Customer CEO's perspective about the CrowdStrike / Microsoft debacle: #£$ KEY POINTS (copied and pasted for quick headline read) -------------------------------------------------------------- - Delta Air Lines CEO Ed Bastian said the massive IT outage earlier this month that stranded thousands of customers will cost it $500 million. - The airline canceled more than 5,000 flights in the wake of the outage, which was caused by a botched CrowdStrike software update and took thousands of Microsoft systems around the world offline. - CEO said the carrier would seek damages from the disruptions, adding, "We have no choice." #technology #releasemanagement #outage #impact https://lnkd.in/ezKHpmAC
Delta CEO says CrowdStrike-Microsoft outage cost the airline $500 million
cnbc.com
To view or add a comment, sign in
-
Last Friday, what was supposed to be an update for all computers across the CrowdStrike network resulted in one of the largest global IT outages in recent history. The outage was caused by an update on the Falcon side, a program that runs on a large number of Microsoft-operated devices. Microsoft reported that approximately 8.5 million devices were affected by the outage, which is less than 1% of their total devices. While the percentage may seem low, the impact on that 1% led to significant disruptions in various sectors. One of the most widely affected was airlines. Delta Air Lines, in particular, is still struggling to recover. From AP News: "The Atlanta-based carrier has canceled more than 6,600 flights since the outage, far more than any other airline, according to figures from FlightAware and travel-data provider Cirium." On Monday Secretary of Transportation, Pete Buttigieg posted to X (formerly Twitter): "The U.S. Department of Transportation has opened an investigation into Delta Air Lines to ensure the airline is following the law and taking care of its passengers during continued widespread disruptions. All airline passengers have the right to be treated fairly, and I will make sure that right is upheld." Check out this Misery Map from FlightAware showing cancelled and delayed flights from early Monday morning through yesterday evening illustrating the widespread impact of this incident. How did the CrowdStrike crash affect you? #travelnews #travelindustry #delta #crowdstrike
To view or add a comment, sign in
-
Breaking News: A major IT outage has struck businesses worldwide, disrupting operations across airlines, financial services, and media groups. From Sydney to London, companies are grappling with significant disruptions, with many employees unable to log in. Airlines like Delta, United, and American Airlines have grounded flights, and airports face delays. The outage, blamed on a security update from CrowdStrike affecting Microsoft Windows, is being called one of the largest in history. 𝗚𝗲𝘁 𝗺𝗼𝗿𝗲 𝗱𝗲𝘁𝗮𝗶𝗹𝘀 𝗮𝘁: https://lnkd.in/ebUVqzGm #ITOutage #TechNews #BusinessDisruption
To view or add a comment, sign in
-
-
How do you avoid this or minimize it in the modern world? Simply invest in your IT/Security infrastructure. Companies just still find it hard to invest in IT/SEC, these should events should be a good reminder to always be evaluating and keeping up with your infrastructure.
Experienced Marketer helping Senior Marketers & Ad Tech CEO’s hone their narratives and enabling skills. Vast experience in data strategy, consent and compliance, privacy tech, digital media, social, search, brand safety
I think CrowdStrike is going to be writing some very big checks in the coming years! This event and it’s aftermath may ultimately lead to new customer diversification strategy that helps to mitigate risk. Having such an enormous market penetration with a technology that is destined to fail or be breached at one point actually catalyzed what will likely be an equity extinguishing event. How to mitigate success driven risk at this scale will be the subject of much introspection and debate over the coming years. Delta Air Lines CEO Says CrowdStrike Tech Outage Cost It $500 Million https://lnkd.in/dnXGHhk7
Delta CEO Says CrowdStrike Tech Outage Cost It $500 Million
wsj.com
To view or add a comment, sign in