Service Issue Tokyo - Linode Kubernetes Engine
Incident
Report for Linode
Postmortem
At 14:41 UTC on October 29th, 2023, we observed a small subset of customer LKE clusters having issues communicating with the API within the Tokyo Datacenter. This caused a lack of visibility for affected clusters in Cloud Manager and impacted autoscaling capabilities.
At 15:51 UTC, we located a misconfigured SSL certificate. This caused the symptomatic API communication issues.
At 15:59 UTC, after correcting the SSL certificate, full service was restored to the impacted LKE clusters.
We then continued monitoring the service until we deemed the issue resolved at 17:32 UTC.
Moving forward, we have identified an area of improvement for our LKE product SSL implementation.
Resolved
This incident has been resolved.
Monitoring
A fix has been implemented and we are monitoring the results.
Investigating
Our team is investigating an issue affecting the Linode Kubernetes Engine (LKE) in Tokyo. Customers may experience issues interacting with clusters via the Cloud Manager at this time, as well as issues auto-scaling existing clusters. We are investigating and will update as new information becomes available.
This incident affected: Linode Kubernetes Engine (AP-Northeast-2 (Tokyo 2) Linode Kubernetes Engine).