What CDK and the Auto Industry Can Teach Us About Critical Infrastructure Security

The CDK ransomware attack was big news, but for most Americans, it wasn’t a big deal because it didn’t affect them personally. Except for dealership employees, the very few who had immediate car buying needs, the larger but still limited groups who’s repairs, service, or parts were delayed (they probably had the roughest time with it), the majority of people out there may have heard or read about the ransomware attack but felt no impacts from it.

What if it had affected millions of Americans in dense cities or large regions? It may surprise you, but the core issues that dealerships and CDK faced and led to the effective ransomware also exist in many of our critical infrastructure systems – things like water, waste, air and rail transportation, pharmacy networks, internet providers and natural gas. What if this ransomware had locked out one of those systems and people were without one or more basic services for the 3 weeks it took CDK to straighten out their mess? We should look at CDK as an opportunity to analyze the deficiencies that allowed the attack to happen, especially focused on the wide effect it had for an entire industry, but remain thankful for the minimal impact to daily life.

CDK should open our eyes to supplier risk

The very pointed effect in this case is that CDK – a single supplier of services in the automotive services space – shut down hundreds (some report thousands) of dealerships across the entire country due to their lapse in cyber protection and policy. Let’s take a quick look at what CDK provides to their clients:

• Sales support (including inventory, cost, and agreements)
• Service support
• Parts support
• IT equipment and support
• Financial institution integration for dealers.

Obviously having such a large amount of dealership operations based on a single platform makes CDK a critical piece to the companies that contracted them. This is our first tie back into critical infrastructure – we also see a large amount of aggregation of services in critical infrastructure sectors. Many of the services have consolidated into a few vendor companies with industry specific knowledge, and through a massive M&A push over the last decade, many of those core vendors have acquired additional tools and services outside their initial core business that expand their reach within a client’s environment.

The next eye-opening piece of this puzzle is that through a limited infection point, most (if not all) of CDKs customers were affected. There are rumors that the attack started from a compromised dealer VPN uplink, as well as rumors it started from an irate support caller reaction. Both are absolutely rumors, as CDK has been very tight lipped, but neither are truly relevant. I’ve spent many years in consulting, which have included both pen-test and incident response – this comes down to one thing – poor system segmentation and management. Regardless of how the initial foothold was gained, the attacker was able to pivot across the CDK infrastructure enough to cause a shutdown of all clients. Similarly in critical infrastructure, we find that vendor data breaches tend to include critical information from multiple utility providers, making the vendors a higher risk that utility provider. In one case of an OSINT security project, we found a company that provided maintenance service to multiple nuclear facilities which had the org chart and phone numbers of the plant executives to help their sales folks stored in an unprotected website. These kinds of finds make everyone cringe.

Finally, and most importantly, we can be pretty certain in our assertion that CDKs disaster recovery plan failed. When the breach was initially announced, CDK said they expected return to operations in 2-3 days. Then on June 20, they announced they had suffered an additional breach, and it would take 2-3 weeks to resume normal operations. If I were in Las Vegas (with some of the closed dealerships at a lot of free time), I would take a solid bet that CDK had tried to initiate their backup recovery, and found that their backups were ineffective because the attacker still had their foothold and root access to reinfect or their backups contained the credentials and exploitable systems that allowed the attacker to regain access in exactly the same manner. Once again, in our tie back to what keeps our country running – in the many years as a cyber consultant, I saw VERY few companies who had ever exercised their Disaster Recovery Plan. Most had it as a paper only document, and had never attempted a simulated tabletop with it. For the few that asked us to do this with them, rarely did they involve a red team. You really need that adversarial approach to get an idea of what recovery looks like when basic plans fail.

QBing from the Armchair

As a security practitioner, here are a things that stood out to me throughout the CDK event. First, CDK shut out everyone during the attack. Their clients, the media, everyone. Of course that makes the internet rumors go wild, but it also does not give anyone confidence in your handling of the situation. Keeping people informed is key to regaining confidence. It also makes my QB role here my interpretation only – we have no evidence either way, so we can only infer based on our experience. Second (related to the first) – there has so far been no information on what PII (if any) from the dealerships has been compromised. Again, being forthcoming is key – it will get out in the inevitable legal discovery, so why not be proactive? Third, the available information that can be found about CDKs infrastructure and VPN to cloud delivery does not show a company that has a modernized security architecture. Finally, all other knowns considered, the biggest failure seems to be lack functioning disaster recovery plan. If we give every other issue a pass of good intentions, this is what made the ransom a lasting event. More discussion on this below.

What has CDK really taught us?

As promised, we should take a few lessons learned to improve our security from the expansiveness of this attack before it happens in an industry that has the potential to cost lives.

1. Know your suppliers and vendors. If they are critical to your operations and service delivery, make them prove to you that they have a good security policy and that they follow their own policy. We use vendor controls assessments for this – it should be standards based, but your assessments can add focus and thorough understanding to areas of data and operations security critical to your business. Most importantly, those results should be supported by validated evidence. Checking ‘yes’ in a security survey should not be enough. SOC2 and other certifications are great, but their generalized approach will not answer questions specific to your relationship.
2. In a similar vein – if they are critical to your business, do not be shy about asking about their Disaster Recovery Plan documentation, results, and last time the plan was exercised. You should have more than just a policy plan that says when service should be restored. Show me you’ve done the work.
3. Focus inward – if you have a critical vendor, your own DRP and tabletop should include losing that vendor as your first action. If it doesn’t, get ready to print out a lot of ‘temporarily closed’ signs for your front doors. Include a red team in your tabletops. I promise you, it’s a lot more fun that way, and you’ll definitely run across things you haven’t thought of. Again, experience from my consulting days is that there is no more cost and time effective incident response activity than doing a DRP tabletop. You don’t want to start learning those lessons when you’re already having a very bad day.

CDK gave us a good wake-up call in a thankfully limited impact (compared to our essential services) bubble. Overall, as consumers and service providers, we all must acknowledge that cyber events are a part of life, and there is no such thing as perfect security. That absolutely does not mean we give up – it means that we should demand – with our pocketbooks – that companies become more transparent in how they protect our information and services that are critical to our lives and businesses. Make all your vendors – new and especially ones you’ve worked with for a long time – PROVE that they are taking steps to protect you from cyber attacks. As citizens, speak up in county level meetings advocating for testing our critical infrastructure (looking at you water districts). As families spend just a little time out of your day/week/year making your own personal disaster plan.