At around 17:00 UTC on November 14th, 2024, we started noticing failures across several data centers (including London, Dallas and Atlanta) after a change made by Akamai. This change was supposed to enable a new feature on host systems that would be transparent to end users. However, this unexpectedly caused some Linodes to fail booting, and those that were already running on affected hosts were unable to apply changes to Cloud Firewall rules.

In order to mitigate the issue, Akamai rolled back the deployed configuration at around 21:19 UTC on November 14th, 2024. In addition, all failed firewall rules were identified and remediated. Customers with affected firewalls were directly contacted.

To prevent this issue from happening in the future, Akamai will be prioritizing the resilience of the underlying service used to invalidate configurations for this infrastructure, defining a better method of configuration testing, and adding tests specific to this type of configuration test failure.

This summary provides an overview of our current understanding of the incident given the information available. Our investigation is ongoing and any information herein is subject to change.