Linode Status

Current Status
Connectivity Issue - Newark (us-east)
Incident Report for Linode

Between 18:14 and 18:20 UTC, two switches in our Newark data center powered off automatically. After further investigation, it was discovered that this was due to temperature issues affecting those switches.

The data center provided temporary replacement cooling for these switches before these switches were power cycled. However, at around 20:55 UTC, the data center's field technicians accidentally powered off the wrong switch. The switch that was powered off handled connectivity for Linode's management network. Due to an architectural redundancy failure, this prevented Linodes in Newark from processing new jobs (power changes, Cloud Firewall updates, etc.).

This management switch was powered back on at 21:20 UTC, but it did not have a configuration in place, and its previous console connection was no longer in use. Linode's network technicians coordinated with remote hands to give it a new console connection in order to supply the switch with a working configuration. The application of this configuration was successful at 23:07 UTC, restoring the management network connectivity.

We have fast-tracked a project to mitigate the network design to prevent this redundancy failure in the future. Additionally, we will be implementing monitoring to alert us when network configuration changes have not been made permanent.

Posted Nov 21, 2023 - 16:43 UTC

We haven’t observed any additional connectivity issues in our Newark data center, and will now consider this incident resolved. If you continue to experience problems, please open a Support ticket for assistance.
Posted Sep 07, 2023 - 00:51 UTC
At this time, we have been able to correct the issues affecting connectivity in our Newark data center. We will be monitoring this to ensure that it remains stable. If you are still experiencing issues, please open a Support ticket for assistance.
Posted Sep 06, 2023 - 23:28 UTC
Our team has identified the issue affecting connectivity in our Newark data center. We are working quickly to implement a fix, and we will provide an update as soon as the solution is in place.
Posted Sep 06, 2023 - 22:37 UTC
Our team is investigating an issue affecting connectivity in our Newark data center. During this time, existing services in Newark are expected to continue operating normally, but changes to services (including but not limited to creating new services, migrations, power state changes, Cloud Firewall changes, etc.) may not function as expected. We will share additional updates as we have more information.
Posted Sep 06, 2023 - 21:29 UTC
This incident affected: Regions (US-East (Newark)).