Linodes failing to boot and apply firewall changes in all Data Centers
Incident
Report for Linode
Resolved
We haven't observed any additional issues causing boot failures for Linodes as well as inability to make changes to Cloud Firewall rules, and will now consider this incident resolved. If you continue to experience issues, please contact us at 855-454-6633 (+1-609-380-7100 Intl.), or send an email to support@linode.com for assistance.
Monitoring
As of 21:19 UTC we have been able to correct the issue causing boot failures for Linodes as well as the inability to make changes to Cloud Firewall rules. We will be monitoring this to ensure that the service remains stable. If you are still experiencing issues and unable to open a Support ticket, please call us at 855-454-6633 (+1-609-380-7100 Intl.), or send an email to support@linode.com.
Identified
As of 19:46 UTC, we have been able to fix the issue causing the inability to boot Linodes and those booted Linodes are able to apply changes to Cloud Firewall rules. We are actively working to fix the Cloud Firewall rules that were deployed during the impact window as they are not applied at the moment. We will share additional updates as we have more information.
Update
To add on to our previous post, customers may have issues with creating/recycling Linode Kubernetes Engine nodes and/or deploying Managed Databases as a result of this issue. Any Cloud Firewalls used along with those services would likely be unable to apply rule changes, as well.
Investigating
At around 17:00 UTC on November 14th after a change we started noticing Host job failures across several Data Centers (including London, Dallas and Atlanta) specifically, some Linodes are failing to boot, and those which fail to boot may also be unable to apply changes to Cloud Firewall rules. Our team is investigating a service issue. We will share additional updates as we have more information.
This incident affected: Linode Manager and API.