Linode Status - Scheduled Maintenance

Scheduled Maintenance - LKE Services

Scheduled Maintenance Report for Linode

Postmortem

Starting at roughly 16:40 UTC on April 26, 2023, several services supporting customer interaction (including the Cloud Manager and Linode API) experienced a fault due to an unexpected increase in backend load. This fault did not affect the stability of customer resources directly, only the management (creation, updating, deletion) of resources.

Customers would have noticed increased latency in jobs processing, and perhaps later as a result of the latency, errors due to backend connections being left in a pending state.

The fault was addressed in two steps:

Firstly, by adjusting the resource configuration of the associated systems.
Secondly, by modifying the services' configuration themselves to increase reliability and scalability.

To prevent these issues moving forward, we are planning out the implementation of improved resiliency and alerting measures on the affected systems.

This issue was resolved around 22:40 UTC on May 16, 2023, and has since been monitored, with no recurrence reported.

Posted Jun 28, 2023 - 20:23 UTC

Completed

The scheduled maintenance is canceled and going to be rescheduled at a later date.

Posted May 26, 2023 - 16:20 UTC

Scheduled

We will be performing scheduled maintenance to the Linode Kubernetes Engine (LKE) service, starting at 12:00 UTC on May 30th, 2023 and completing at 00:00 UTC on May 31, 2023. During this time, we will be upgrading and rolling restarting the calico-node daemonset and calico-controller in each LKE cluster. This will result in brief intermittent intercluster communication interruptions. If you are experiencing other issues with your cluster during this time please open a Support ticket from the Linode Manager.

Posted May 23, 2023 - 20:37 UTC

This scheduled maintenance affected: Linode Kubernetes Engine (US-East (Newark) Linode Kubernetes Engine, US-Central (Dallas) Linode Kubernetes Engine, US-West (Fremont) Linode Kubernetes Engine, US-Southeast (Atlanta) Linode Kubernetes Engine, CA-Central (Toronto) Linode Kubernetes Engine, EU-West (London) Linode Kubernetes Engine, EU-Central (Frankfurt) Linode Kubernetes Engine, AP-South (Singapore) Linode Kubernetes Engine, AP-Northeast-2 (Tokyo 2) Linode Kubernetes Engine, AP-West (Mumbai) Linode Kubernetes Engine, AP-Southeast (Sydney) Linode Kubernetes Engine).