On June 17, 2024, at approximately 7:33 UTC, we were alerted by multiple reports from customers of an issue with Object Storage buckets not being displayed in the Cloud Manager. Investigation into this issue began immediately, and our incident response team was then engaged.
At approximately 12:04 UTC, our administrators determined that the problem was being caused by an overloaded process tied to Object Storage. After restarting this process, the immediate impact to customers was mitigated. Further investigation into the root cause of the issue continued.
At 17:11 UTC, a software fix targeting the source of the problem was implemented. It was then decided that a period of extended monitoring would be required to ensure a full resolution.
On June 20th, 2024, at 18:39 UTC, our administrators concluded their monitoring of the situation with no further issues detected, and the incident was considered resolved.
In order to prevent similar issues in the future, our administrators have made improvements to our alerting process and have implemented additional configuration changes to strengthen the reliability of our service.
We are continuing to monitor to ensure that the impact has been fully mitigated.
We will provide the next update as we make progress.
We will provide the next update as we make progress.
Customers facing errors while accessing Object Storage from Cloud Manager are encouraged to retry their request. If you continue to experience any problems, please open a Support ticket for further assistance.
We will continue to monitor to ensure that the impact has been fully mitigated.
We are actively investigating the issue, and, will provide another update as we progress.