Hi all, I'd like to share a feature I've been working on and get your feedback: node groups (node pools) for CloudManaged CKS clusters.
PR: https://github.com/apache/cloudstack/pull/14594 Documentation PR: https://github.com/apache/cloudstack-documentation/pull/699 Motivation: today all the worker nodes of a CKS cluster share the same service offering and template, so running different workloads (GPU, high memory, dedicated nodes for some teams) requires separate clusters. Node groups are a common feature of managed Kubernetes offerings (GKE, AKS, EKS) that users expect from CloudStack as well. What the PR adds: - Node groups with their own service offering, template, Kubernetes labels and taints, managed with new create/list/update/scale/deleteKubernetesClusterNodeGroup APIs and a new Node groups tab in the UI - Every cluster gets a default node group holding the worker nodes of the cluster size (created on upgrade for existing clusters); scaleKubernetesCluster keeps working and applies to it, so existing clients and the Terraform provider are not affected - Per-node labels and taints on top of the ones of the node group (updateKubernetesClusterNode API) - Per node group autoscaling settings; the cluster autoscaler is deployed with one --nodes argument per autoscaled node group - Operations on different node groups of a cluster can run concurrently Open point I'd especially like feedback on: the upstream CloudStack provider of the Kubernetes cluster autoscaler only supports a single node group identified by the cluster ID. Autoscaling node groups other than the default one therefore needs an autoscaler supporting node groups, configurable with a new global setting. I'm considering contributing that support to the kubernetes/autoscaler CloudStack provider as well, and would welcome opinions on the approach. The change has been tested with unit tests and manually on a KVM zone. Reviews and comments are very welcome. Thanks, Artem Grinberg
