Linode Status

Current Status

Service Issue - All Services - US-EAST (Newark)

Incident Report for Linode

Postmortem

On July 27, 2025, at approximately 08:30 UTC, Akamai services in the Newark, NJ (us-east) data center experienced a critical outage due to overheating of infrastructure. This overheating was triggered by a utility power failure at the facility, which resulted in the loss of HVAC functionality. Although the data center itself remained operational, the failure of the cooling system caused elevated temperatures that directly impacted Akamai hardware, leading to the shutdown of core services. The outage affected Linode Compute Instances (commonly referred to as 'Linodes'), Object Storage, NodeBalancers, and Linode Kubernetes Engine (LKE) within the Newark region. Additionally, internal dependencies on Newark infrastructure caused degraded performance for LKE services in other regions, including Dallas, Fremont, Sydney, Tokyo 2, Toronto, and Washington DC.

‌In response, we initiated mitigation efforts that included replacing failed network hardware, rerouting traffic away from Newark, and migrating impacted workloads to backup systems where possible. Restoration efforts were executed in phases once temperatures stabilized, prioritizing services based on severity and customer impact. While some services began recovering on July 28th, full recovery was completed by July 29th at 16:22 UTC.

We are currently conducting a comprehensive post-incident review to identify opportunities to improve our resilience. This includes auditing cross-region service dependencies that allowed an issue isolated to Newark to affect services elsewhere. We are also evaluating architectural improvements to better isolate services by region and increase fault tolerance. In parallel, we are reviewing our monitoring and alerting systems to improve early detection of cooling-related risks and ensure faster mitigation. Our commitment to transparency, accountability, and service reliability remains steadfast as we work to strengthen our systems and prevent future occurrences.

‌This summary provides an overview of our current understanding of the incident given the information available. Our investigation is ongoing and any information herein is subject to change.

Posted Aug 04, 2025 - 23:54 UTC

Resolved

We haven’t observed any additional issues with the Newark, NJ (us-east) region, and will now consider this incident resolved. If you continue to experience problems, please open a Support ticket for assistance.
Posted Aug 01, 2025 - 15:39 UTC

Monitoring

At this time we have been able to correct the issues affecting all services in our Newark, NJ (us-east) data center. This included Linode Kubernetes Engine (LKE), Linode compute instances, Block and Object Storage, and additional services. Some downstream impact was observed in other data centers due to interdependencies with Newark, and those services have also been fully restored. We will be monitoring this closely to ensure continued stability. If you continue to experience problems, please open a Support ticket for assistance.
Posted Jul 28, 2025 - 16:42 UTC

Update

We are continuing to work on restoring full functionality to Linode Kubernetes Engine (LKE) and Linode compute instances in our Newark (us-east) data center. Our teams are actively engaged in recovery efforts and closely monitoring progress.

We appreciate your continued patience as we work to fully resolve this incident.
Posted Jul 28, 2025 - 14:51 UTC

Update

Mitigation efforts are continuing with our subject matter experts actively working to restore the remaining services. We will continue to provide updates as progress is made.
Posted Jul 28, 2025 - 10:50 UTC

Update

Mitigation efforts are continuing with our subject matter experts actively working to restore the remaining services. We will continue to provide updates as progress is made.
Posted Jul 28, 2025 - 08:50 UTC

Update

Mitigation efforts are continuing with our subject matter experts actively working to restore the remaining services. We will continue to provide updates as progress is made.
Posted Jul 28, 2025 - 05:16 UTC

Update

Our SMEs are working towards bringing up the remaining services online. The situation is improving. We will provide further updates as further progress is made.
Posted Jul 28, 2025 - 02:29 UTC

Update

Our SMEs continue to work bringing our services back online, and we will provide an update as progress is made.
Posted Jul 27, 2025 - 23:54 UTC

Update

Our SMEs continue to work bringing our services back online. We have been able to correct the issues impacting LKE in our Dallas, Fremont, Sydney, Tokyo 2, Toronto and Washington data centers. Issues impacting Block Storage, Object Storage, and Nodebalancers in Newark were also restored per previous updates . We are still working to mitigate impact to LKE and individual Linode instances in Newark and will provide updates as progress is made.
Posted Jul 27, 2025 - 22:42 UTC

Update

At this time we have been able to correct the issues affecting Nodebalancer as of 19:00 UTC. We will be monitoring this to ensure that it remains stable.
Efforts to bring our services back continue, we will provide updates as progress is made.
Posted Jul 27, 2025 - 21:36 UTC

Update

Our SMEs continue to work bringing our services back online, and we will provide an update as progress is made.
Posted Jul 27, 2025 - 21:00 UTC

Update

We are continuing to work on a fix for this issue.
Posted Jul 27, 2025 - 19:56 UTC

Update

We are continuing to work on a fix for this issue.
Posted Jul 27, 2025 - 19:56 UTC

Update

At this time we have been able to correct the issues affecting LKE in our Dallas, Fremont, Sydney, Tokyo 2, Toronto and Washington data centers. Issues in Newark are still being worked at this moment.
We can also confirm Block Storage has recovered as of 14:13 UTC, as well as Object Storage at 17:52 UTC. We will be monitoring this to ensure that it remains stable.
Efforts to bring our services back continue, we will provide updates as progress is made.
Posted Jul 27, 2025 - 19:16 UTC

Update

Our SMEs continue to work bringing our services back online, and we will provide an update as progress is made.
Posted Jul 27, 2025 - 17:37 UTC

Update

In regards of the LKE impact caused in the identified data centers due to this issue, customers would have inability to provision new clusters and degraded ability to recover from control plane failures for already running clusters while this issue is active. Autoscaling of existing node pools is not impacted except for Newark data center. We continue to work bringing our services back online, and we will provide an update as soon as progress is made.
Posted Jul 27, 2025 - 16:16 UTC

Update

Impact to LKE services has been confirmed to have also extended to our data centers in Dallas, Fremont, Sydney, Tokyo 2, Toronto and Washington due to the interaction with our data center in Newark. We continue to work bringing our services back online, and we will provide an update as soon as progress is made.
Posted Jul 27, 2025 - 14:24 UTC

Identified

Our team has identified the issue affecting connectivity in our US-EAST (Newark) data center. The issue is related to heating/cooling complications in the data center due to a power outage . The power outage has been fixed and we are working quickly to bring our services back online, and we will provide an update as soon as the solution is in place.
Posted Jul 27, 2025 - 13:59 UTC

Update

Our team continues investigating an issue affecting connectivity in our US-EAST (Newark) data center. During this time, users may experience unavailability issues for all services deployed in this data center. We will share additional updates as we have more information.
Posted Jul 27, 2025 - 13:29 UTC

Update

Our team is investigating an issue affecting connectivity in our US-EAST (Newark) data center. During this time, users may experience unavailability issues for all services deployed in this data center. We will share additional updates as we have more information.
Posted Jul 27, 2025 - 12:34 UTC

Investigating

Our team is investigating an emerging service issue affecting All Services in US-EAST (Newark). We will share additional updates as we have more information.
Posted Jul 27, 2025 - 10:08 UTC
This incident affected: Regions (US-East (Newark)), Block Storage (US-East (Newark) Block Storage), NodeBalancers (US-East (Newark) NodeBalancers), Backups (US-East (Newark) Backups), Object Storage (US-East (Newark) Object Storage), Linode Kubernetes Engine (US-East (Newark) Linode Kubernetes Engine, US-Central (Dallas) Linode Kubernetes Engine, US-West (Fremont) Linode Kubernetes Engine, US-IAD (Washington) Linode Kubernetes Engine, CA-Central (Toronto) Linode Kubernetes Engine, AP-Northeast-2 (Tokyo 2) Linode Kubernetes Engine, AP-Southeast (Sydney) Linode Kubernetes Engine), Metadata Service (US-East (Newark)), and Longview.