We encounter issues on SharedSQL and CloudDB servers. The issue started at about 18:00 CEST. More information in a few minutes…
Update(s):
Date: 2020-10-20 18:05:49 UTC 17:18 CEST: A physical host hosting a PCI database crashed, because of a failed PSU (note that this PSU is being replaced). This database contains data like where the VM are (on which physical hosts), their status… Usually, this database is not that critical, as a VM can run without it. The Openstack actions are just not available. But live migrations were ongoing on some VM (the ones listed above). The database loss lead to VM UP on 2 hosts, at the same time. So the impact is that the DBMS (your DBMS, the one you're using as a customer) that still had connections on the source VM have lost their transactions, if any. The ones connected to the destinations have not.
We apologize for any inconvenience that this issue could have caused you.
Date: 2020-10-20 17:49:09 UTC sharedsql071.sql006:mysql710 UP and running.
Date: 2020-10-20 17:42:55 UTC The incident is over but on sharedsql071.sql006, mysql710 instance.
Date: 2020-10-20 17:36:49 UTC The incident is over. More information on the root cause in a few minutes.
Date: 2020-10-20 17:36:01 UTC Impacted VM (the list is exhaustive):
- clouddb010.sql006
- sharedsql071.sql006
- clouddb011.sql007
- clouddb048.sql007
- clouddb017.sql020
- sharedsql043.sql021
- sharedsql011.sql022
Posted Oct 20, 2020 - 17:29 UTC
This incident affected: Web Hosting || Datacenter GRA (Cluster002, Cluster003, Cluster006, Cluster007, Cluster011, Cluster012, Cluster013, Cluster014, Cluster015, Cluster017, Cluster020, Cluster021, Cluster023, Cluster024, Cluster025, Cluster026, Cluster027, Cluster028, Cluster029, Cluster030, Cluster031).