Bjorn Watland’s Post

Everyone is sharing their hot takes on this _draft_ PM from CrowdStrike, who am I to go against the wind? 1. You can have better testing, but you can't have best testing. You can't even fully validate that all combinations of flags to /bin/ls are valid. The best we can hope for is some level of reassurance that we've caught the big ones. 2. If you have two paths to deployment, one of them is measured and canaried and follows all of those other nice practices, and the other one is meant to ship something now because OMG, it should be obvious which one needs to have more thorough vetting and very close observability before going out the door. 3. If you're going to continue to allow live updates of things that impact your kernel, you're going to keep having these incidents. eBPF, I'm looking at you, too. https://lnkd.in/dY_Zkbw9

Kibana Machine Learning – Security Alert Mechanism Security Incident Detection is the Key Feature of the Kibana Machine Learning tools. When creating your custom job & dashboard it’s important to get the settings right to ensure you properly configure the job, threshold value setting to make use of ML models and select the analyzed parameters. Kibana ML Security Incident Detection provides a great visual analysis interface enabling the operator to review time-series data with critical and warning level highlights mapped over time in different color coding which is easy to understand by the L1 team. #mss #msp #mssp https://lnkd.in/dmdbvfHG

Very nice project that can help when building threat/response plans with different EDR platforms.

Better information security means reaching across organizational lines, such as deploying configuration hardening and passwordless authentication. All the teams involved are like clusters of neurons, performing tasks that span across teams is like establishing new connections between those clusters. Creating, tuning, and restructuring those connections creates strategic advantage and it doesn't matter where that skill starts.

Check out this quick reference guide of daily logs used by our Expel SOC team + analysts! It's designed to make life a bit easier. Get your copy here🙂: https://okt.to/Pw8KhI

Crowdstrike PIR has been released. While our company was not affected, this truly could have happened to anyone. To the many companies that were affected, sysadmins everywhere, we look forward to your continued progress and sleepless nights and the recovery of those companies who were brought down. Check out the PIR for an interesting read. We have a lot to learn for our own environments along with the clients we manage. This just stresses the fact of performing test in sandbox environments, and pushing updates much like these with staggered deployment strategies to minimize affected assets and users. I’ve even seen some talks about utilizing different tools in different levels of infrastructure to minimize risk if an incident much like this happens but that’s a whole other can of worms. https://lnkd.in/gnUKum5p

I hope Crowdstrike serves as a great reminder of the risk of releasing on a Friday with insufficient testing time. Great analysis about the incident in this blog, wild that an empty file could do so much damage. A big learning for me over the past year has been the importance of testing your code with the assumption that it's already broken: you just haven't seen how yet. Adding tests to cover scenarios such as "whoopsies this important service is unavailable" and know exactly how things will continue executing or gracefully fail are a great way to make an overall system more anti-fragile. https://lnkd.in/gTDnJ3kT

I've been reading the various Crowdstrike post-mortems and root cause analysis. Like many recent high-profile outages, this looks like a "misconfiguration" This analysis caught my attention: On July 19, 2024 at 04:09 UTC, as part of ongoing operations, CrowdStrike released a sensor configuration update to Windows systems. This configuration update triggered a logic error resulting in a system crash and blue screen (BSOD) on impacted systems. Read my thoughts about the high cost of misconfigurations and how to prevent them: https://hubs.la/Q02JfXrX0 #devops #configmanagement #misconfigurations #crowdstrike

🚨 Heads up! CrowdStrike's Falcon update caused Windows crashes. They've fixed it and shared the scoop on how to prevent it. 🛠️ Dive into the details here: https://lnkd.in/gGPGs4kZ

Tip of the Day, Thursday, September 5 Review Access Logs Use a tool like Splunk, Elastic, or Graylog, Inc. to analyze access logs. Set it to alert you if someone tries to access your system from an unusual location or if there are multiple failed login attempts. Review access logs regularly to identify potential security incidents and take corrective actions. Implement logging and monitoring tools to automate log analysis and generate actionable insights. Train employees on the importance of access log reviews and how to recognize and report suspicious activities.