Beginning at 22:35 UTC on August 1st, 2023, an issue with Linode's backend database impacted performance of the Linode Cloud Manager, as well as the Linode API. This resulted in customers experiencing slow response times in accessing their accounts or being able to issue API calls. This did not impact performance of existing services.
At 22:40 UTC, our database administrators were made aware of the incident and began investigating immediately. A bug within the backend of the database was discovered and mitigated. At 22:47 UTC, our systems began to show signs of recovery, and we began to receive reports of an improvement in the situation. By 23:40 UTC, we were able to confirm that the database had returned to expected performance.
It was determined by our incident response team that an extended investigation would be launched to ensure stability of the service moving forward. After completing this initiative, the incident was considered resolved at 14:13 UTC on August 3rd, 2023.
To prevent recurrences of this issue, we have upgraded the backend of our database services. Since this was completed, we have not witnessed any further issues with the components affected here.