At approximately 12:50:00 PM EST on February 23, 2018, utility power was interrupted to our Fremont data center. The UPS systems provided power for their respective areas while 7 of 8 generators started and began to come up to operating temperature. During this phase, one of the generators servicing Linode’s infrastructure failed to start. UPS power for our area was exhausted at 13:01:00 EST causing a subset of Linode hardware (and the customer instances on them) to be powered off unexpectedly and become unreachable.
Utility power was restored at approximately 13:18:00 EST. At that time, Linode engineers worked to bring our infrastructure and affected customer instances back online. All Linode infrastructure and customer instances were online by approximately 16:50:00 EST, and the incident was deemed resolved.
Further diagnostics determined that the generator’s autostart controller had failed, preventing a successful start and subsequent transfer of generator power. The colocation site’s vendor was engaged and a replacement card was installed.
We do not anticipate any further issues with the Fremont facility at this time. Thank you for your patience and understanding. We apologize for any inconvenience this outage has caused.
If you are still experiencing connectivity issues, please reach out to our Customer Support Team for assistance.