Hey! I think this question should have been sent to the user mailing list instead of dev :)
In any case: 1: Yes the autoscaler logic is triggered in the reconcile loop, but that just means that it is triggered every X seconds and only if there is no other thing to do from the operator like performing the upgrade. This is to ensure that the autoscaler does not interfere with upgrades or the other way around. 2: The parallelism changes / overrides are stored in the autoscaler and are always applied on the FlinkDeployment/SessionJob resource to make sure it is preserved during restarts/upgrades etc. 3: It doesn't check that there are enough resources before scaling however when used with in-place scaling the adaptive scheduler will not stop the previous job until the resources are acquired. 4: You cannot use this autoscaler together with HPA/VPA. So you can either use the autoscaler (which is much better in our opinion) or use the HPA with reactive mode but that doesn't work well the kubernetes operator. In any case I highly recommend to simply try and play around with this locally, that will answer most of your questions. Cheers, Gyula On Wed, Nov 6, 2024 at 2:11 AM santhosh venkat <santhoshvenkat1...@gmail.com> wrote: > Hi, > I have a few questions about the Flink OSS Kubernetes operator. Apologies > if they’re basic. > > 1. It seems like the auto-scaler logic is triggered as part of the > operator > reconciliation loop > < > https://github.com/apache/flink-kubernetes-operator/blob/main/flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/reconciler/deployment/AbstractFlinkResourceReconciler.java#L132 > >. > Does this mean the auto-scaler is expected to be applied as part of job > deployment or when the state of any of the watched K8s objects is > changed > over time. > 2. From the implementation, it seems like OSS flink K8s operator changes > the parallelism of flink-operators in the job depending upon processing > rate and catchup rate configurations. Just wondering if these > parallelism > changes are preserved across job restarts/deployments or if the > parallelism > configured in the job for different flink operators is used upon > restart. > 3. Just wondering how the auto-scaler checks if the kubernetes cluster > has sufficient capacity before increasing the parallelism of > flink operators. > 4. Are users expected to use HPA or VPA in conjunction with the > auto-scaling module in Flink K8s operator. Or are these two unrelated to > each other? > > Thanks. >
