gyfora opened a new pull request, #484: URL: https://github.com/apache/flink-kubernetes-operator/pull/484
## What is the purpose of the change This is the initial implementation of the autoscaler module as described in https://cwiki.apache.org/confluence/display/FLINK/FLIP-271%3A+Autoscaling As descirbed in the FLIP and ML discussion, the initial implementation will work best for sources that implement the new unified source metrics (pendingRecords) and use the new source interface (that publishes busyTime metrics). We have mostly tested this with Kafka sources, which provide the most extensive functionality at the moment. The actual scaling executions rely on some feautres currently only present in Flink 1.17-snapshot : - https://github.com/apache/flink/commit/23ce2281a0bb4047c64def9af7ddd5f19d88e2a9 (must have) - https://github.com/apache/flink/commit/a7fdab8b23cddf568fa32ee7eb804d7c3eb23a35 (good to have) This means that you can use the latest 1.17 snapshot docker image (as shown in the example) or that you have to backport these fixes to your local 1.15/1.16 images. The autoscaler itself should work with other Flink versions as long as the backports are present. ## Verifying this change In addition to the unit tests provided, we have also verified the autoscaler in different environments on small and large scale Flink jobs and we confirmed the intented behaviour. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changes to the `CustomResourceDescriptors`: no - Core observer or reconciler logic that is regularly executed: yes ## Documentation - Does this pull request introduce a new feature? yes - If yes, how is the feature documented? JavaDocs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
