Major Australian superannuation provider UniSuper, which serves 647,000 members, has revealed that Google’s Cloud service accidentally deleted primary data and all its backups.
However, the $135 billion ($89.56 billion) fund was able to recover from the mishap, only because it had backed up its data on another service.
According to the company’s incident log, the downtime started on May 2, and a full restoration of services did not occur until May 15.
UniSuper’s website has published a joint statement from UniSuper CEO Peter Chun, and Google Cloud CEO Thomas Kurian regarding the issue.
“Google Cloud CEO, Thomas Kurian has confirmed that the disruption arose from an unprecedented sequence of events whereby an inadvertent misconfiguration during provisioning of UniSuper’s Private Cloud services ultimately resulted in the deletion of UniSuper’s Private Cloud subscription.
“This is an isolated, ‘one-of-a-kind occurrence’ that has never before occurred with any of Google Cloud’s clients globally. This should not have happened. Google Cloud has identified the events that led to this disruption and taken measures to ensure this does not happen again.”
Explaining the delay in restoring the data, the statement goes on to say, “UniSuper had duplication in two geographies as a protection against outages and loss. However, when the deletion of UniSuper’s Private Cloud subscription occurred, it caused deletion across both of these geographies.”
It seems Google Cloud’s safeguards against account deletion failed, and the only option was a restore from a separate cloud provider.
Engineer Says A Simple Bug Caused the Outage
While Google hasn’t explained precisely what happened, software engineer Daniel Compton has analysed the outage and concluded the problem dated back to when the account was created.A bug caused the entry of an incorrect value, which meant the private cloud was created with a one-year subscription rather than a perpetual one.
“After one year passed, Google Cloud dutifully deleted the private cloud,” he explains.
“The press release uses vague language, obscuring the technical details of what happened ... [but given] it is now clear that this was indeed a Google Cloud bug, I remain puzzled why the communication over this incident was so bad. By their silence, Google Cloud let millions of people think that they had deleted a company’s entire Google Cloud environment.”
Mr. Compton expresses the hope that APRA, Australia’s superannuation regulator, would investigate the incident.
