At approximately 20:57 UTC on September 24, 2024, we began to receive customer reports regarding issues with Linode compute instances failing to boot. During this time, any host-level tasks such as boot, reboot, and other similar jobs were stuck in a pending state until the issue was identified and resolved on the backend.
Our investigation traced the problem to the deployment of an updated host software affecting a subset of our fleet across multiple regions. This update unintentionally caused disruption, preventing Linode compute instances from executing jobs as expected. Once the root cause was identified, we promptly rolled back the software update, which immediately resolved the issue.
We are actively reviewing our deployment processes to prevent similar incidents in the future. As part of our ongoing efforts, we will implement additional safeguards for hypervisor image rollouts to minimize the risk of further disruptions.
This summary reflects our current understanding of the incident based on the available information. As our investigation continues, further updates may follow if new details emerge.