RackTop Systems’ Post

My three takeaways from the Crowdstrike outage: 1) Security teams and infrastructure teams need to work more closely together and understand how all the pieces work together. I often feel that security teams don't understand how the infrastructure works and operates. This gives them a false sense of security. The infrastructure team doesn't pay enough attention to how the security tools work and what they need which leaves gaps in protection and poor user experience. 2) Architecture matters almost more than anything. If you have a good architecture, you can survive faults, mistakes, attacks and outages. If you have an IT architecture that doesn't match the needs and operations of your organization, it will be catastrophic. With the right architecture things like a bad update can be rolled back in minutes or a malware infection can be contained in seconds. 3) Defense in depth and diversification are paramount. In this case an outage was caused by a software update to one application and we saw how critical infrastructure was brought to its knees. However, an adversary could use this same method of a software update to use your security software against you! They could exfiltrate data silently from organizations across the globe. A nation state could launch this denial of service attack at the same time as a kinetic strike. So there needs to be layers of security and tools that can provide visibility and check on the other security tools. Everyone likes standardization and simplification, but there are times where the risk outweighs the benefits. Do you disagree? https://tcrn.ch/3Lx6xiW

What did we learn from the CrowdStrike incident? I am typically a big supporter of designing solutions where security is a first class citizen. In other words, secure by design. However that doesn’t mean, we should apply all security methods to any component in an IT solution. This could lead to getting almost nothing done, or (as we see with the CrowdStrike ‘incident’) to impact availability without a reasonable threat. There seems to me 2 things at play in cybersecurity (based on the fear of an attack): 1) protection against the ‘unknown unknowns’ and 2) ‘not on my watch’. Somehow we need to get past this fear fuelling us to implement draconian security measures. From what I observed the CrowdStrike incident created most havoc as it made end points unusable. I totally support protecting end points used as work stations where significant ‘ad hoc’ manipulations of data takes place. But end points that serve as a terminal to server based applications, display panels, kiosks, or ATMs? In other words where processing on the end point is 100% predictive? Do we really need to bombard these infrastructure components with end-less security patches? If am confident that architects of transportation or financial transaction processing systems have designed and implemented solutions on reliable operating environments that won’t go down on a ‘kernel panic’. And if they do, high availability kicks in to prevent a larger outage. But these same architects (rightfully) assume that ‘terminals’ are stable end points. And if something affects an end point, it is an outlier and very likely local. The CrowdStrike incident (actually a disaster) should make us rethink how and where we implement cybersecurity. The fact it is claimed to be an approved software update, doesn’t mean it is without risk. And let us not make the mistake that this is a black swan event. PS on both my personal and work computer I have disabled any auto updates. Not sure how effective on my work computer, but my personal computer has not seen a kernel panic in its 15.5 years of operation. Go figure. (Hint: it hasn’t Windows as the operating system.)

Faulty CrowdStrike update causes major global IT outage, taking out banks, airlines and businesses globally Businesses across the world are reporting IT outages, including Windows “blue screen of death” errors on their computers, in what has already become one of the most widespread IT disruptions in recent years. The outage — linked to a software update from popular cybersecurity firm CrowdStrike — has affected computers running Microsoft Windows at organizations across various sectors, including airlines, banks, retailers, brokerage houses, media companies, and railway networks. The travel sector seems to be one of the hardest hit, based on online chatter. CrowdStrike’s chief executive, George Kurtz, confirmed in a post on X that a “defect” in a content update for Windows hosts had caused the outage, and Kurtz ruled out a cyberattack. He added that the firm was rolling out a fix and that Mac and Linux hosts were not affected. “CrowdStrike is actively working with customers impacted by a defect found in a single content update for Windows hosts. Mac and Linux hosts are not impacted,” Kurtz noted on X. “This is not a security incident or cyberattack. The issue has been identified, isolated and a fix has been deployed. We refer customers to the support portal for the latest updates and will continue to provide complete and continuous updates on our website. We further recommend organizations ensure they’re communicating with CrowdStrike representatives through official channels. Our team is fully mobilized to ensure the security and stability of CrowdStrike customers,” said Kurtz. A post on CrowdStrike’s support forums (which are only accessible with a login) also acknowledged the issue early on Friday, saying the company had received reports of crashes related to a content update. CrowdStrike said the crash reports were “related to the Falcon Sensor” — its cloud-based security service that it describes as “real-time threat detection, simplified management, and proactive threat hunting.” See more: https://lnkd.in/gZAy2fau

Lessons from the CrowdStrike Outage Hey everyone, I wanted to share some reflections on a recent event that has significant implications for all of us in the IT and cybersecurity fields. Last week, we witnessed what some experts are calling the "largest IT outage in history," caused by a software update error from a leading cybersecurity firm, CrowdStrike. This incident disrupted thousands of flights, affected countless businesses, and brought home some critical lessons about the fragility of our digital infrastructure. As someone deeply involved in IT and cybersecurity through my work at Armoureye, I see this as a humbling reminder of how vital our roles are in maintaining and protecting the digital lifelines of countless organizations. Here are a few takeaways that I think are worth considering: 😎 Diversify Your IT Providers: Relying on a single provider can be convenient, but it also means putting all your eggs in one basket. Diversification can help mitigate risks and ensure more robust resilience. 😎 Regular Software Audits: Mistakes happen, even to the best. Conducting frequent and thorough audits of software updates can help catch errors before they cause widespread issues. 😎 Have a Solid Incident Response Plan: It's not just about preventing problems, but also about being prepared to respond quickly and effectively when they do occur. A well-defined incident response plan is essential. 😎 Communicating Transparently: During times of crisis, clear and honest communication with clients and stakeholders is crucial. It helps manage expectations and maintain trust. 😎 We believe in the power of partnerships. By collaborating with Managed Service Providers (MSPs) and cybersecurity product vendors, we can offer comprehensive, reliable solutions tailored to our clients' unique needs. Thank you for taking the time to read this. Let's continue to support each other and share our knowledge to make the digital world a safer place for everyone. Feel free to reach out if you have any questions or if there's anything we can help with. https://lnkd.in/e5kr4isw

IT teams scramble to recover from CrowdStrike incident as officials warn of ‘risks of consolidation'. Key takeaways: 1. The technology outages instigated by a flawed CrowdStrike software update are a vivid demonstration of the risks tied to technology sector consolidation. This incident, which impacted multiple government entities, services, and companies, underscored the potential for widespread and significant disruptions stemming from a single company's misstep. 2. The fallout acts as a wake-up call for the necessity to restore resilience and redundancy back into vital systems. As global organizations more frequently rely upon a limited select group of tech firms, a single-point of failure can lead to cascading ramifications, revealing the fragility of these concentrated systems. 3. The recovery process offers cybercriminals an opportunity for exploitation. Situations like these, where systems and services are down, tend to invite a spike in criminal activity, such as phishing, as threat actors take advantage of the chaos and vulnerabilities presented in system restorations. Learn more by visiting The Record from Recorded Future News: https://lnkd.in/edatnaJT

A faulty software update issued by security giant CrowdStrike has resulted in a massive overnight outage that’s affected Windows computers around the world, disrupting businesses, airports, train stations, banks, broadcasters, and the healthcare sector. Key Takeaways for New Yorkers: 1. Critical Infrastructure Vulnerability: Even robust security solutions can have critical vulnerabilities. 2. Business Continuity Plans: Ensure your business continuity plans are up-to-date to handle unexpected disruptions. 3. Regular Backups: Regularly backup data to minimize the impact of outages. 4. Timely Updates and Patches: Balance the need for timely updates with thorough testing to prevent widespread issues. Stay informed and prepared to mitigate potential cybersecurity risks. https://lnkd.in/eC6T9pPx

📉 Lessons from CrowdStrike's Historic Outage Last Friday's IT meltdown serves as a critical reminder for all of us in the tech and cybersecurity sectors. CrowdStrike’s software update debacle, affecting nearly 9 million Windows computers, disrupted hospitals, airlines, banks, and even UK doctors’ offices, forcing them back to pen and paper 📎. It's a stark lesson in the importance of rigorous testing and incremental rollouts. This incident underscores the necessity of diversified risk management strategies. The primary catalyst was a flawed Windows update from CrowdStrike’s Falcon platform, which led to the infamous "blue screen of death" and continuous reboots. Unlike external cyber-attacks, this failure was homegrown, highlighting the vulnerabilities lurking within our own systems. Critical to this fiasco was the choice of timing—a Friday, exponentially increasing the risk of unnoticed issues over the weekend 📅. Reflecting on this, it's clear organisations must re-evaluate their vendor dependency and internal resilience strategies. This isn’t just about having contingency plans but ensuring those plans are drilled and actionable. As Apratim "AP" Purakayastha pointed out, proactive and thorough business continuity planning is vital. Let this be a wake-up call: Our greatest threats might not be external hackers but flaws within our own operations. Let's take this seriously, plan better, and strengthen our collective IT resilience 💪. #RiskManagement

"A Deep Analysis of Global IT Outage Y2K24" This report provides an in-depth analysis of the major global IT outage known as “Y2K24” that occurred on July 19, 2024, due to a software update error from CrowdStrike. The report details the timeline of the incident, identifies the root cause, provides a technical breakdown, analyzes the impact and discusses the lessons learned from this event. https://lnkd.in/eREu3WER #Y2K24 #CrowdStrikeIncident #ITOutage #Cybersecurity #CyberSecurityNews #CyberSecurityUpdate

Just how vulnerable is our digital world? Find out in our latest blog post where we explain everything about the recent global IT outage, its causes, impacts across industries, and crucial lessons on cyber resilience. Don't miss out on this eye-opening read. 👉 Read more on our website: https://lnkd.in/gMPjwzDR #CyberSecurity #ITOutage #TechNews #MuiaConsulting

It seems that this is the largest IT outage of all time. Its cause is a software update by #Cybersecurity firm CrowdStrike that brought down Microsoft and numerous other businesses linked to them. The interesting question is who is responsible and how can avoid this happening again? Who is accountable to the passengers stuck in airports (or the numerous other affected across the globe) ? Is it the airlines, the airports, Microsoft, CrowdStrike or should it be regulators? The European Union has introduced legislation on resilience ( #DORA) but it only covers only financial services - what else is happening on legislating dependability? Are CEOs accountable or just the IT people? Should businesses be required to have plan Bs to avoid similar situations?