We have detected slowness at GRA1 and GRA3 (cluster020, cluster021, cluster023, cluster025 cluster026 cluster027, cluster028 , cluster029, cluster030)
and also more recently at GRA2 (cluster003).
Start time: Mon June 07 2021 05:15
Severity level : Medium
User impact : Websites might be slow or not accessible temporarily
Technical impact (DC/geographical area) : GRA1, GRA2 and GRA3
Thank you for your understanding
OVHcloud Technical team
Update(s):
Date: 2021-06-09 15:43:31 UTC Incident resolved
Date: 2021-06-09 14:47:25 UTC The main issue seems to be resolved since 16:40 Europe/Paris
We keep the network under surveillance and will close this incident if everything stays stable.
Date: 2021-06-09 13:45:26 UTC Filerz are now up.
Date: 2021-06-09 13:21:26 UTC The improvement on the network side reported above did not improve the desired effect.
We will have to cut the access to the following filerz for about ten minutes :
- filerz1005
- filerz1012
- filerz1013
- filerz1020
- filerz1027
- filerz1036
Date: 2021-06-09 10:13:42 UTC We make changes on the network side in order to investigate and solve the problem.
Date: 2021-06-08 22:24:26 UTC We are still working on this case. An external escalation concerning a network equipment is ongoing.
Date: 2021-06-08 21:08:39 UTC We are still working on this case.
Date: 2021-06-08 18:19:47 UTC More network tests are currently run.
Date: 2021-06-08 16:26:52 UTC We are still running network tests.
Impacted websites are still mostly on clusters 028 and 030.
Date: 2021-06-08 13:22:25 UTC We managed to reproduce the network issue, we now isolate the network equipments to find the faulty one
Date: 2021-06-08 12:28:25 UTC The intervention on cluster030 is currently finished, we keep investigating on network
Date: 2021-06-08 10:48:17 UTC We are operating an intervention on outgoing traffic on cluster30
Date: 2021-06-08 02:48:43 UTC Rebooting network equipments did not fix the issue, although performance should be better now.
Impacted websites are mostly on clusters 028 and 030.
Date: 2021-06-08 01:05:52 UTC Network isolation did not succeed. We are rebooting a main core network equipment. No further impact is expected.
Impacted websites are mostly on clusters 003, 015, 028 and 030.
Date: 2021-06-07 22:40:22 UTC We are still working on this case. Impacted websites are mostly on clusters 003, 028 and 030.
Date: 2021-06-07 20:38:11 UTC Network issues have emerged as a potential cause.
We are still working to resolve this situation.
Thank you for your patience
Date: 2021-06-07 19:12:01 UTC We are still working on this case
Date: 2021-06-07 16:52:39 UTC We are still working on this case
Date: 2021-06-07 15:54:04 UTC Websites hosted in GRA2 district might also be impacted, but incident is mostly located at GRA1 and GRA3.
Date: 2021-06-07 15:39:33 UTC We are still working on this case
Date: 2021-06-07 14:42:47 UTC Websites on cluster030 will be unavailable 5 minutes starting 16:45 Europe/Paris
Date: 2021-06-07 14:28:19 UTC Websites using storage equipment filerz 1020 will be unavailable 10 minutes starting 16:30 Europe/Paris
Date: 2021-06-07 14:00:37 UTC We keep investigating
Date: 2021-06-07 10:53:49 UTC We keep working on it
Posted Jun 07, 2021 - 09:44 UTC
This incident affected: Web Hosting || Datacenter GRA (Cluster002, Cluster003, Cluster006, Cluster007, Cluster011, Cluster012, Cluster013, Cluster014, Cluster015, Cluster017, Cluster020, Cluster021, Cluster023, Cluster024, Cluster025, Cluster026, Cluster027, Cluster028, Cluster029, Cluster030, Cluster031).