Starting 19:00 UTC on Jul 17, 2023, a performance issue with Linode's backend database impacted performance of the Linode Cloud Manager as well as the Linode API. This resulted in customers experiencing slow response times, or being unable to access their accounts or issue API calls. This incident did not impact access to existing services.
The team was made aware of the issue at 19:09 UTC, and our database administrators immediately began investigating. At 19:35 UTC, the administrators started clearing out database queries causing the contention and performance issues. By 20:00 UTC, we were able to confirm the database was once again performing as expected.
The root cause of the performance issues was the use of locking facilities in our database layer, which have been modified to mitigate this type of problem.