ConnectWise’s Post

The recent #CrowdStrike update affected 8.5 million #Windows systems, highlighting critical lessons for the future. Key takeaways include the importance of robust testing procedures, comprehensive data backup strategies, and diversification of services. Learn how Dropsuite can enhance your business continuity and prevent data loss: https://ow.ly/bwCL50SKyW3 https://ow.ly/cGrz50SKyW1

Crowdstrike outage: Hello World, Should you hear from anyone impacted by the Crowdstrike outage, feel free to reach out to our team for help or share the following solution with them:   1. Boot into Safe Mode or Windows Recovery Environment: - Restart your system and press F8 repeatedly before Windows start to boot. Alternatively, you can use the Windows Recovery Environment. 2. Navigate to the CrowdStrike Directory: - Go to C:\Windows\System32\drivers\CrowdStrike. 3. Delete the Problematic File: -Locate the file matching “C-00000291*.sys” in the CrowdStrike directory and delete it 4. Restart Normally - Once deleted, restart your system normally. Hope this helps! For more detailed instructions, Here is an article of what happened:

Let's consider the scale of this incident. 8.5 million Windows devices comprise merely 0.5% of the Windows install base, and less than 0.05% of all connected devices. There can (and will) be future disruptions orders of magnitude more severe than this one.

Update: CrowdStrike has issued a statement that the issue causing the outage on Windows hosts was due to a content update, and a fix has been deployed. CrowdStrike also assures that the outage was not due to cyberattack, and its systems are operating normally. For clients who are still facing issues with the hosts, please refer to the workarounds listed in the statement. We understand the frustration caused to our customers and to all the tech teams who have been working hard to resolve this, and hope that everyone affected has the issue resolved soon, if not already. Meanwhile our SOC and engineering teams are on hand to support affected clients and assist with the restoration process. https://lnkd.in/gtZQ5TD4

🔔 Important Update: CrowdStrike Outage   If you or any of your co-workers are experiencing the "Blue Screen of Death" or issues stemming from the CrowdStrike update - keep reading! What Happened? On July 19, Microsoft machines running CrowdStrike Falcon encountered a BSOD (Blue Screen of Death) and got stuck in a restarting state. CrowdStrike has isolated the issue and issued a fix. The CrowdStrike CEO confirmed that this is not a security incident. Key points: 🖥️ Affected Systems: Only Windows devices are impacted. 🌐 Broader Impact: This issue may affect companies that do not use CrowdStrike Falcon, as many software and platform vendors are experiencing service degradation, including Microsoft, LogMeIn, Jonas, USPS, and more. 🔄 Patch Update: CrowdStrike temporarily ceased patch deployment and released an updated patch to rectify the issue as of 1:27 AM EDT. ❗️WorkSmart's Recommendation: If you are running CrowdStrike, avoid rebooting your systems for the time being to allow the updated patch to apply. For more info, you can refer to CrowdStrike's event page here: https://lnkd.in/gtZQ5TD4 P.S. If you're looking for a committed IT partner to help when incidents like this arise - contact us today! #CrowdStrike #worksmart #WorkSmartIT #keepingITsecure

My condolences to anyone sitting with Windows endpoints and Crowdstrike right now. I hope you are able to recover ASAP. For visibility, this is the recovery steps on an endpoint that is stuck in a BSOD crash loop: 1. Boot Windows into Safe Mode or the Windows Recovery Environment 2. Navigate to the C:\Windows\System32\drivers\CrowdStrike directory 3. Locate the file matching “C-00000291*.sys”, and delete it. 4. Boot the host normally. This is inofficial information and if you have a direct line to Crowdstrike, please talk to them directly for better assistance. For people out of the loop, here is the news: https://lnkd.in/d3pN8esV Given the complexity of the recovery instructions (entering safe mode often requires physical access to the endpoint and a keyboard attached), I will assume that this will take some organisations days to recover all endpoints from.

System administrators around the world have been fixing BSOD in Windows PC fleets for the third day due to an error in CrowdStrike software At Delta, the Bitlocker security option is activated on all devices, up to kiosks, so the engineers of the technical support team have to manually work with each PC. System administrators, network specialists and engineers of thousands of companies online around the world continue to manually fix the BSOD in Windows PC fleets for the third day due to a global failure caused by an error in the CrowdStrike software update. The logical error of the Falcon Sensor cyberattack protection system manufactured by the American information security company CrowdStrike affected the IT infrastructure of many banks, government agencies, airports, and enterprises from various industries around the world. System administrators and engineers in shifts of 10-16 hours on the floor, on their knees, on stairs, in gyms, hospitals, factories and police stations bring PCs, smart panels, ticket kiosks, servers and laptops back to life manually, remote connection does not work. You need to boot the system into Safe mode and run some commands or work with the registry. If the PC's disk is protected by BitLocker encryption, then you need to find and enter the BitLocker recovery key on each system, and then continue fixing the CrowdStrike update until all computers in the organization are working. #news

What!? Windows 3.1 / Windows 95 computers immune to the CrowdStrike worldwide IT outage? Oh the irony: Southwest Airlines still uses these ancient operating systems for staff scheduling systems, and the reason they are unaffected, is simply because they don't get software updates. LSA Digital's very own Danielle Linn is still stuck in Atlanta because other airlines use modern, updated software. The WRONG lesson here, is to use old, out of date technology. It's all to easy to think, "oh it's on the intranet, no one will hack it", but from a Zero Trust cybersecurity principles perspective, we have to worry about security INSIDE the perimeter, not just outside. A BETTER lesson is to consider a 1-2 punch combo: (1) An Application Portfolio Rationalization to find the highest-criticality, highest-risk technology and "AT LEAST" properly maintain (patch) it, or otherwise modernize it (and you can ask Keith Mangold and Anthony Phillips about that) https://lnkd.in/e9eaXEic (2) Make sure those patches are trusted via a Secure Software Supply Chain principles - e.g., Software Bill of Materials (SBOM) are free from malware, and preferably, certification that the organization providing the patch has QA processes in place for assurance around things like sufficient testing -- e.g., progressive rollout of patches ("smoke testing") (and you can ask Ben Amaba, PhD, PE, CPIM, LEED AP and Cate Richards from Sonatype about the second punch) https://lnkd.in/eu7Y5vzJ #sbom #devsecops #softwaresupplychain #ApplicationRationalization #ITPortfolioManagement #EnterpriseArchitecture

Global Outage Alert: Windows BSOD Crisis Following CrowdStrike Update – Recovery Steps & Qualys Assurance: On Friday, July 19, 2024, morning, reports surfaced globally of Microsoft Windows operating system users encountering the infamous Blue Screen of Death (BSOD) following the latest update from CrowdStrike. This widespread issue has severely impacted critical services, including telecommunications, banking, airline and railway operations, supermarkets, hospitals, and major news networks. CrowdStrike has outlined a four-step […]