City of Jacksonville’s Post

After the Microsoft / CrowdStrike issue caused what's being called the 'largest IT outage in history,' let's explore the importance of disaster recovery planning. Visit Rentacomputer.com to learn more: https://lnkd.in/gYgfKFhj Or check out our blog to read up on the current outage: https://lnkd.in/g8T4pd27

I'm currently on an 14 hour Amtrak train ride from Washington DC to Atlanta, GA because the flight that I was supposed to take is still experiencing system issues due to the Crowdstrike patch/Microsoft upgrade that caused a system outage worldwide on July 9th. Why are some companies systems up and running and others are not. The more companies depend on these cloud based systems the more vulnerable they are. Being in the tech industry for over 25 years the questions that I have include: 1. Are we testing their infrastructure and having multiple fail safes in place? 2. Are there rollback mechanisms in place and the ability to run in "safe mode"? 3. Are there multiple data centers to reroute traffic? 4. Does the company have a contingency plan that is exercised at least once a year Normally these questions don't happen until after a system outage of this magnitude occurs. Companies need to take some of their corporate profits and invest in a airtight contingency plan to mitigate future risk. Ok it's 430am. 5 hours left to go. 🤔

Despite even the best efforts to achieve 911 center diversity, failure points can still exist, leading to potential network and system outages. That’s why 911 centers need to be constantly vigilant by monitoring and assessing every network, system, device, and connection that it uses to provide service to emergency responders and citizens. MCP’s Milton Schober outlines this topic in the second installment on our two-part blog series discussing the importance of 911 center diversity. Learn more: https://bit.ly/3JKlRYx

When natural disasters or cyberattacks strike, communications are often the first to be impacted. In response, Cisco NERVs are on 24/7 standby for rapid deployment, providing advanced and secure communications for emergency responders. Learn how in our latest blog. http://cs.co/6041Y5OKB #Disasterrecovery #Techblogs

🚨Recent 911 Outages: A Reminder of Communication Backups. Recent 911 service disruptions in states like Texas and Nebraska highlight the importance of having backup communication systems in place for emergencies. While 911 remains vital, relying solely on it can leave communities vulnerable. Multimodal communication offers redundancy and accessibility for people with varying communication needs. What does this mean? Imagine receiving critical alerts during an outage via text message, email, or voice call—all through a reliable Emergency Mass Notification System (EMNS) platform. A robust EMNS system leverages all of the tools available when needed most. Learn more: https://ow.ly/GNHG30sBTfb

❓ How did the CrowdStrike outage happen, and why was it so severe❓ TL;DR: On July 19, 2024, CrowdStrike’s corrupted update caused global panic, affecting everything from airlines to hospitals. It didn't just hold up operations; it completely shut critical systems down. So what went wrong? Let's break it down: Kernel Drama: A corrupted "Rapid Response Content" file bypassed validation, leading to the Blue Screen of Death for millions of Windows users. Infinite Loops: The corrupted update tossed computers into an infinite reboot loop. Fixing this required physical access, which was a nightmare with the Blue Screen of Death. Missed Canary Deployments: A staggered or canary deployment could have minimized the damage. However, their rapid deployment of the corrupted file to all endpoints made things worse. Rollback Issues: Yes, there was a rollback plan, but considering the widespread impact and the Blue Screen of Death - this process was more complicated than the plan was prepared for. How do you avoid causing an outage like this? In this ✨ special edition ✨ blog from Bryan Finster, we're reminded that even the smartest can make mistakes. Avoiding an outcome like this is not just about canary deployments and rollback plans. It's a call to constantly stress-test our processes, learn from others' mistakes, and ALWAYS have a backup plan. To dig into our analysis, check out Bryan's full postmortem blog post here: https://shorturl.at/8V5dN

When natural disasters or cyberattacks strike, communications are often the first to be impacted. In response, Cisco NERVs are on 24/7 standby for rapid deployment, providing advanced and secure communications for emergency responders. Learn how in our latest blog. http://cs.co/6040YIl3C #Government #GovernmentNow #Disasterrecovery #Techblogs

When natural disasters or cyberattacks strike, communications are often the first to be impacted. In response, Cisco NERVs are on 24/7 standby for rapid deployment, providing advanced and secure communications for emergency responders. Learn how in our latest blog. http://cs.co/6041Ykp8W #Government #GovernmentNow #Disasterrecovery #Techblogs

When natural disasters or cyberattacks strike, communications are often the first to be impacted. In response, Cisco NERVs are on 24/7 standby for rapid deployment, providing advanced and secure communications for emergency responders. Learn how in our latest blog. http://cs.co/6048Y5LDE #Government #GovernmentNow #Disasterrecovery #Techblogs