Avnish Tripathi’s Post

Let this serve as a lesson: replicate backups outside of your cloud provider's infrastructure. The individual responsible for having done so in this case is a hero. This situation could have easily been a disaster. https://lnkd.in/eCEQq7rY

This did not get the same extent of coverage that cyber-attacks have garnered in the past. It is labelled by Google as a very unusual, one off, unlikely to be repeated event, but if it can happen once, it can happen again. A reminder that being with a big cloud service provider that is perceived as well-run is not necessarily a guarantee of security. UniSuper is a big client for Google, so they would have got the full support and attention of Google. If it was a smaller client, would Google be as responsive and accommodating? Kudos to UniSuper for having a back-up with a separate provider. A simple step that a lot of organisations don't necessarily take. #google #dataloss #UniSuper #backup https://lnkd.in/g3RiCzBs

Google Cloud provisioning misconfiguration leads to UniSuper's business disruption for a week Google Cloud mistakenly deletes UniSuper's private cloud account, causing a week-long outage for over half a million members. UniSuper CEO reassures members that the outage wasn't a cyber-attack and no personal data was compromised, placing the blame squarely on Google's cloud service. UniSuper and Google Cloud CEOs issue a joint apology, acknowledging the frustration and disappointment caused by the incident. The deletion stemmed from an unprecedented misconfiguration during provisioning, a scenario never encountered by Google Cloud before. Despite the setback, UniSuper's collaboration with Google Cloud and backup measures with another provider lead to an extensive recovery of core systems, ensuring minimal data loss and service restoration. Business continuity management? https://nis2.management #misconfiguration #google #cloud #unavailability #businesscontinuity

All organization has to think about this. When we had onprem servers before cloud, there were 3-4 datacenters as backup. Below incident could be a learning to have backup cloud provider for your application? Read below. https://lnkd.in/e8dWFNBM

Cloud providers are not infallible. Between the big names and the layers of abstraction, providers like #AWS, #Azure, #GCP, and others can provide a false sense of security and stability. Nuking a big account is certainly an outstanding eff-up on Google's part, but less remarkable "misconfigurations" are not altogether uncommon. I've personally been involved in multiple interventions where cloud infrastructure went pear-shaped with impacts ranging from mildly annoying to feeling like we barely averted disaster. The more you have to lose the less it hurts to have a backup plan in a different basket. https://lnkd.in/eJd6isjd

"The cloud is someone else's server" really comes to life when Google Cloud wipes a customer's account. Congratulations are in place for the team at UniSuper who worked hard to recover their systems and data, while keeping their customers informed as much as possible. They clearly had proper business continuity and disaster recovery policies in place, and someone was wise enough to have secondary backups with a third-party provider. The official statement relies on vague language and obscure actors like “an unprecedented sequence of events whereby an inadvertent misconfiguration during provisioning of UniSuper’s Private Cloud services ultimately resulted in the deletion of UniSuper’s Private Cloud subscription.” It is a real shame that Google Cloud has not released a postmortem report, like other cloud providers do. Those of us who work in technology deserve a proper report with all the technical details, explaining what happened, how it happened and what have they done to prevent this from happening again. https://lnkd.in/gGTrXiPK

I suspected something like this. If you must have a hotswap backup in the one cloud provider, do it on a different account. For 4 reasons: 1. It enforces a full separation between the two instances, no sneaking an interdependency around the side. 2. You can test in isolation, knowing loading on one will not impact the other. So you could use your backup instance for pen testing or do A/B deployments and discover problems before they hit production. 3. It removes the account as a single point of failure. 4. It proves your Infrastructure As Code (IaC) framework works across multiple deployments as a truly parameterised layer. #unisuper #cybersecurity #Australia #technology #google

The recent issue at UniSuper is a reminder of how important it is to have a strong backup plan. Thankfully, UniSuper had backups with another service provider, which helped them get back on track. This situation also shows us that using multiple cloud architecture may help preventing similar problem if one of the cloud service stops working ... 🤞🏻 //“This is an isolated, ‘one-of-a-kind occurrence’ that has never before occurred with any of Google Cloud’s clients globally. This should not have happened. Google Cloud has identified the events that led to this disruption and taken measures to ensure this does not happen again.” While UniSuper normally has duplication in place in two geographies, to ensure that if one service goes down or is lost then it can be easily restored, because the fund’s cloud subscription was deleted, it caused the deletion across both geographies. UniSuper was able to eventually restore services because the fund had backups in place with another provider.// #resilience #multicloud #oneofakindoccurrence #incident https://lnkd.in/gmDK4fPt

Can anyone hear you scream in the Google Cloud? A company managing $135B in assets had its Google Cloud account deleted, along with all of its data, even though the company used two different zones in the Google Cloud. Fortunately, the company also had additional backups in a non-Google location and was eventually able to recover its data. Let's put all of our stuff in one vendor's cloud and hope for the best, even if you are using two different zones for data protection. I'd be interested in seeing the post-mortem on this one, which was never supposed to happen...until it did. https://rebrand.ly/p81er93 #GoogleCloud #majorfail #accountdeleted #datadelted #whodoneit #publiccloud

As everything rapidly moves to the cloud this is scary. One of the main reasons to move to the cloud protection and survivability of your data is completely negated.