pashtet04 opened a new issue, #151:
URL: https://github.com/apache/superset-kubernetes-operator/issues/151
## What happened?
When `autoscaling` is configured for scalable components, the operator
creates HPA resources, but the operator also keeps reconciling the target
Deployments back to `replicas=1`.
Observed:
- HPA reports `AbleToScale=True` and `1 current / 2 desired`.
- Deployments are repeatedly reverted to `replicas=1`.
- Extra pods created by HPA start and are immediately terminated.
- This repeats continuously.
Expected:
When `autoscaling` is configured, the operator should not fight HPA for
`Deployment.spec.replicas`. HPA should manage scaling, as documented:
> When HPA is configured, the `replicas` field is ignored (HPA manages
scaling).
The current behavior appears to come from the Deployment reconciliation path
doing a full Deployment spec update. Even when the desired spec has
`Replicas=nil`, Kubernetes defaults `Deployment.spec.replicas` to `1`, so each
reconcile overwrites the replica count that HPA just set.
## Environment
- Operator version: `0.1.0`
- Helm chart: `superset-operator` `0.1.0`
- Operator image: `ghcr.io/apache/superset-kubernetes-operator:0.1.0`
- Kubernetes server version: `v1.32.1`
- Install method: Helm chart, deployed by Argo CD
Kubernetes server version:
```json
{
"major": "1",
"minor": "32",
"gitVersion": "v1.32.1",
"gitCommit": "e9c9be4007d1664e68796af02b8978640d2c1b26",
"gitTreeState": "clean",
"buildDate": "2025-01-15T14:31:55Z",
"goVersion": "go1.23.4",
"compiler": "gc",
"platform": "linux/amd64"
}
```
## Reproduction
Redacted CR shape:
```yaml
apiVersion: superset.apache.org/v1alpha1
kind: Superset
metadata:
name: superset
namespace: superset
spec:
webServer:
autoscaling:
minReplicas: 2
maxReplicas: 4
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
celeryWorker:
autoscaling:
minReplicas: 2
maxReplicas: 6
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
```
Live CR after removing the `replicas` fields:
```console
$ kubectl get superset superset -n superset -o
jsonpath='{.spec.webServer.replicas}{"\n"}{.spec.webServer.autoscaling}{"\n---\n"}{.spec.celeryWorker.replicas}{"\n"}{.spec.celeryWorker.autoscaling}{"\n"}'
{"maxReplicas":4,"metrics":[{"resource":{"name":"cpu","target":{"averageUtilization":70,"type":"Utilization"}},"type":"Resource"}],"minReplicas":2}
---
{"maxReplicas":6,"metrics":[{"resource":{"name":"cpu","target":{"averageUtilization":70,"type":"Utilization"}},"type":"Resource"}],"minReplicas":2}
```
HPA state:
```console
horizontalpodautoscaler.autoscaling/superset-web-server
Deployment/superset-web-server cpu: <unknown>/70% 2 4 1
horizontalpodautoscaler.autoscaling/superset-celery-worker
Deployment/superset-celery-worker cpu: <unknown>/70% 2 6 1
```
Deployment state after operator reconciliation:
```console
superset-web-server replicas=1
superset-celery-worker replicas=1
```
Relevant events:
```console
Normal SuccessfulRescale horizontalpodautoscaler/superset-web-server
New size: 2; reason: Current number of replicas below Spec.MinReplicas
Normal SuccessfulRescale horizontalpodautoscaler/superset-celery-worker
New size: 2; reason: Current number of replicas below Spec.MinReplicas
Normal Started pod/superset-web-server-...
Started container superset
Normal Killing pod/superset-web-server-...
Stopping container superset
Normal Started pod/superset-celery-worker-...
Started container superset
Normal Killing pod/superset-celery-worker-...
Stopping container superset
```
Operator logs around the churn:
```console
{"level":"info","ts":"2026-06-24T03:07:04Z","msg":"Reconciled
component","controller":"superset","controllerGroup":"superset.apache.org","controllerKind":"Superset","Superset":{"name":"superset","namespace":"superset"},"namespace":"superset","name":"superset","reconcileID":"11c087c4-d8d5-44a7-922d-4878c89bb579","component":"web-server","name":"superset-web-server","operation":"updated"}
{"level":"info","ts":"2026-06-24T03:07:04Z","msg":"Reconciled
component","controller":"superset","controllerGroup":"superset.apache.org","controllerKind":"Superset","Superset":{"name":"superset","namespace":"superset"},"namespace":"superset","name":"superset","reconcileID":"11c087c4-d8d5-44a7-922d-4878c89bb579","component":"celery-worker","name":"superset-celery-worker","operation":"updated"}
{"level":"info","ts":"2026-06-24T03:07:07Z","msg":"Reconciled
component","controller":"superset","controllerGroup":"superset.apache.org","controllerKind":"Superset","Superset":{"name":"superset","namespace":"superset"},"namespace":"superset","name":"superset","reconcileID":"795cb464-40dc-4c47-8c3e-b64b7956373b","component":"web-server","name":"superset-web-server","operation":"updated"}
{"level":"info","ts":"2026-06-24T03:07:07Z","msg":"Reconciled
component","controller":"superset","controllerGroup":"superset.apache.org","controllerKind":"Superset","Superset":{"name":"superset","namespace":"superset"},"namespace":"superset","name":"superset","reconcileID":"795cb464-40dc-4c47-8c3e-b64b7956373b","component":"celery-worker","name":"superset-celery-worker","operation":"updated"}
{"level":"info","ts":"2026-06-24T03:07:18Z","msg":"Reconciled
component","controller":"superset","controllerGroup":"superset.apache.org","controllerKind":"Superset","Superset":{"name":"superset","namespace":"superset"},"namespace":"superset","name":"superset","reconcileID":"017f5d19-f7db-4dbf-8f4b-641d7c03f8d0","component":"web-server","name":"superset-web-server","operation":"updated"}
{"level":"info","ts":"2026-06-24T03:07:18Z","msg":"Reconciled
component","controller":"superset","controllerGroup":"superset.apache.org","controllerKind":"Superset","Superset":{"name":"superset","namespace":"superset"},"namespace":"superset","name":"superset","reconcileID":"017f5d19-f7db-4dbf-8f4b-641d7c03f8d0","component":"celery-worker","name":"superset-celery-worker","operation":"updated"}
```
No acceptable workaround found. Lowering `minReplicas` to `1` only hides the
immediate minimum-replica churn; any HPA scale-up is still reverted back down
by the operator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
