On December 19, 2024, at approximately 6:28 UTC, a new Block Storage cluster was brought online earlier than anticipated due to a process gap. As a result, the cluster was deployed before it was fully prepared to handle customer workloads. When customers attempted to provision new volumes in ORD, the system automatically selected this new cluster, as it was the least full. Unfortunately, this led to intermittent provisioning failures and, in some cases, caused Linodes with attached volumes to encounter boot or restart issues, resulting in boot loops.

The first impact to external customers was observed at 4:23 UTC on December 20, 2024. Upon identifying the issue, we took immediate action to mitigate the impact by marking the affected cluster offline at 12:45 UTC. This action prevented any further customers from being impacted by new Linode provisioning onto the faulty cluster.

To resolve issues for customers who were already affected, we removed the Block Storage volumes created on the faulty cluster at 14:45 UTC. These volumes contained no customer data but were causing issues with the affected Linodes.

This summary provides an overview of our current understanding of the incident given the information available. Our investigation is ongoing and any information herein is subject to change.