andrijapanicsb opened a new issue, #14212: URL: https://github.com/apache/cloudstack/issues/14212
## Relationship to existing issue Follow-up to #13970. #13970 correctly tracks the hard-coded `0.0.0.0/0` source used by CKS for node SSH and also identifies the Kubernetes API and VPC ACL variants. This follow-up defines the cluster-level API/UI contract, lifecycle reconciliation, network-mode behavior, and rule ownership needed to solve the problem consistently rather than only replacing one constant. ## Problem CKS-managed ingress to cluster management endpoints cannot be configured per cluster: - node SSH firewall rules are created with `0.0.0.0/0`; - the TCP/6443 Kubernetes API firewall path is also unrestricted; - VPC ACL rules are created without an explicit `cidrlist`, so the API default is used; - SG-enabled zones use one account-wide `CKSSecurityGroup-<account UUID>` with unrestricted SSH and 6443 ingress; - create/list APIs and the UI have no cluster property representing the desired policy. Manual narrowing is not a supported workaround. `KubernetesClusterScaleWorker.scaleKubernetesClusterIsolatedNetworkRules()` calls `removeSshFirewallRule()`, which identifies a rule mainly by its public port / related PF destination. It then calls `setupKubernetesClusterIsolatedNetworkRules()`, which recreates the rule with `0.0.0.0/0`. Node-add follows the same unrestricted provisioning path. Since an existing firewall rule's CIDR list is immutable, the fix needs a controlled rule replacement workflow. The current removal helpers also risk selecting unrelated user-managed rules that happen to use the same public IP and ports. Observed on current `main` at analysis time: `602d9ec3e03e6350b55dd1349118ea2ccb766cca`. ## Proposed feature Add an editable Cloud-managed CKS cluster property: ```text managementaccesscidrlist=10.20.0.0/16,192.0.2.40/32,2001:db8:1200::/48 ``` In the first version, the same list controls source access to: - node SSH, including the public SSH port-forward range; - the Kubernetes API on TCP/6443. Required API/UI behavior: - add `managementaccesscidrlist` to `createKubernetesCluster`; - add an async `updateKubernetesClusterAccess` API; - return the canonical list and its enforcement mode in `KubernetesClusterResponse`; - add **Management access networks** to the create form and an edit action in cluster details; - validate IPv4/IPv6 CIDRs, trim, deduplicate, and reject malformed or empty lists; - show an explicit warning for `0.0.0.0/0` and `::/0`; - warn that the policy must permit the source address used by the Management Server for CKS SSH lifecycle operations. Add an inherited account-scoped CSV configuration such as: ```text cloud.kubernetes.cluster.management.access.cidrs ``` It should default to `0.0.0.0/0` for compatibility. When the create parameter is omitted, resolve the owner account's inherited account/domain/global value and snapshot it on the new cluster. Later configuration changes should affect new clusters only. Existing clusters without a stored value should retain the legacy effective value `0.0.0.0/0` until explicitly updated. ## Network-specific behavior ### Isolated network Apply the list to CKS-owned ingress firewall rules on the cluster public/source-NAT IP for the SSH public ports and TCP/6443. Port-forwarding and load-balancer rules remain translation/backend objects; the firewall is the source-policy layer. ### VPC isolated tier Apply the list to the CKS-created ingress ACL entries for TCP/22 and TCP/6443. A restricted policy cannot be enforced with the immutable default-allow ACL. In that case, reject create/update with a clear request to use a custom ACL. Preserve the existing default-deny validation. ### SG-enabled zone Use a deterministic per-cluster security group with ingress TCP/22 and TCP/6443 from the configured CIDRs. Do not edit the current account-wide group for one cluster, because that would change every CKS cluster in the account. Existing clusters using the account-wide group need a safe stopped-cluster migration to a per-cluster group, or a clear rejection if the VM security-group associations cannot be changed safely. This branch is conditional on the zone actually being security-group-enabled; a shared network must not be assumed to have security-group enforcement merely because it is shared. ### Shared/routed direct-access network without enforcement If CKS has no SG, firewall, or ACL layer that can enforce the list, report enforcement as external and reject a restricted create/update request. Do not claim that a stored CIDR list is enforced. ## Lifecycle and ownership requirements Persist the desired policy as cluster state (the existing `kubernetes_cluster_details` table is sufficient for the list). Do not infer desired CIDRs from currently installed rules. Track CKS-created firewall, ACL, PF, and LB resources by explicit cluster ownership/identity. Port-only matching is not ownership. Create/update and every lifecycle path must use the stored list: - create and start; - stop/start; - scale up/down; - add/remove node; - upgrade when rules are recreated; - delete cleanup. CIDR update must be idempotent and retryable. Because firewall/ACL CIDR lists are immutable, it should replace only the old CKS-owned rules, verify the new rules, then persist the new desired policy. A failed update must not silently leave the persisted policy different from the effective network rules. ## Acceptance criteria 1. A user can set one or more CIDRs when creating a Cloud-managed CKS cluster. 2. An authorized user can edit the list later via API and UI. 3. List/get APIs return the canonical list and enforcement mode (`FIREWALL`, `VPC_ACL`, `SECURITY_GROUP`, or `EXTERNAL`). 4. Isolated-network SSH and API firewall rules use the exact stored list. 5. Custom VPC ACL entries for SSH and API use the exact stored list. 6. New SG-enabled clusters in the same account can have different policies. 7. Restricted policies are rejected when the selected network cannot enforce them. 8. Scale and add/remove-node preserve the exact list and never restore `0.0.0.0/0`. 9. Unrelated user-created rules on the same ports are never revoked by CKS. 10. Existing clusters remain backward compatible until explicitly updated. 11. Unit/integration tests cover isolated, VPC, SG-enabled, and unenforceable direct-access modes. ## Scope clarification This controls only management ingress created by CKS for SSH and the Kubernetes API. It does not replace `Service.spec.loadBalancerSourceRanges` or the CloudStack Kubernetes provider's source-CIDR annotation for arbitrary Kubernetes `Service` objects. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
