Linode Status

Current Status
Degraded Network Performance in London
Incident Report for Linode
Postmortem

On Sunday April 24th at approximately 7:22am EDT -4 GMT, our network monitoring systems alerted us to packet loss reaching some destinations in our London datacenter. The loss was upstream to Linode and the issue was immediately reported to our upstream IP transit provider Telecity. Telecity responded saying the packet loss was cosmetic and was not affecting production traffic. Linode engineers pushed back on Telecity, and it was eventually identified that an upstream interface to Linode was down causing congestion on the “B” side of Linode’s network. Linode engineers were able to shift some traffic to the “A” side of the network which cleared the packet loss. Telecity was eventually able to identify and fix the down link on their network and normalize our capacity on April 29th, the down link was attributed to a DWDM issues between facilities.

Due to the duration of the outage and lack of communication coming from Telecity Linode asked that Telecity conduct an investigation of the trouble. Linode is satisfied with the findings of the investigation and confident that a prolonged event of this nature will not occur in the future.

The following is a snip from the Telecity investigation

"Following our investigation we have identified that there were failures to communicate and provide updates to Linode throughout the issue, which led to Linode having to chase for information. This has been addressed internally and the teams involved have been dealt with to ensure that this does not reoccur. The issue has also highlighted process improvements that are currently being reviewed by the management team. Moving forward all DMDW issues will be escalated by Support to the IP Support team for further investigation. Equinix sincerely apologise for any inconvenience this incident may have caused."

Posted May 16, 2016 - 21:30 UTC

Resolved
We've been monitoring the network performance in London for the last 24 hours. Network performance has returned to normal, and has remained consistent during our monitoring. As such, we'll now consider this matter resolved. We'll provide a post-mortem on our status page as soon as possible.
Posted Apr 26, 2016 - 13:50 UTC
Identified
Our Network Engineers have identified the issue and are in the process of resolving it.
Posted Apr 25, 2016 - 11:58 UTC
Update
We have an open line of communication with our upstream providers, and we're working towards a solution.
Posted Apr 24, 2016 - 18:43 UTC
Investigating
We're further investigating reports of networking issues in London, and we'll keep this page posted with any new information we may find.
Posted Apr 24, 2016 - 16:28 UTC
Monitoring
We have worked with our upstream provider, connectivity is returned to normal and we will continue to monitor. If you are experiencing any issues please submit a Support Ticket so that we may further assist you.
Posted Apr 24, 2016 - 15:03 UTC
Investigating
We are currently aware of degraded network performance in our London location. We have contacted our upstream provider and currently investigating this matter further.
Posted Apr 24, 2016 - 11:44 UTC
This incident affected: Regions (EU-West (London)).