On March 27, 2024, at approximately 23:04 UTC, a customer report indicated an inability to deploy images from our Paris data center to our Osaka data center during cross-data center migration, cloning, or image deployment processes, due to a network connectivity issue.
Our network engineering team initiated an investigation at around 23:28 UTC on March 27, 2024, to determine the root cause of the network failure. The investigation revealed that border router control policies were causing a drop in legitimate network traffic.
The border router policies were promptly updated and tested at approximately 01:00 UTC on March 29, 2024, restoring reliable network connectivity between Osaka and Paris.
To prevent similar incidents in the future, our network engineering team will conduct an audit of packet control policies at other data centers to ensure they are appropriately configured.