Hi All! I would like to start a discussion thread regarding the structure of the Kubernetes Operator <https://github.com/apache/flink-kubernetes-operator/blob/main/flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/controller/FlinkDeploymentController.java> controller flow. Based on some recent PR discussions we have no clear consensus on the structure and the expectations which can potentially lead to back and forth changes and unnecessary complexity.
*Background* In the initial prototype we had a very basic flow: 1. Observe flink job status 2. (if observation successful) reconcile changes 3. Reschedule reconcile with success/error This basic prototype flow could not cover all requirements and did not allow for things like waiting until Jobmanager deployment is ready etc. To solve these shortcomings, some changes were introduced recently here <https://github.com/apache/flink-kubernetes-operator/pull/21>. While this change introduced many improvements and safeguards it also completely changed the original controller flow. Now the reconciler is responsible for ensuring that it can actually reconcile by checking the deployment and ports. The job status observation logic has also been moved into the actual reconcile logic. *Discussion Question*What controller flow would we like to have? Do we want to separate the observer from the reconciler or keep them together? In my personal view, we should try to adopt a very simple flow to make the operator clean and modular. If possible I would like to restore the original flow with some modifications: 1. Validate deployment object 2. Observe deployment and flink job status -> Return comprehensive status info 3. Reconcile deployment based on observed status and resource changes (Both 2/3 should be able to reschedule immediately if necessary) I think the Observer component should be able to describe the current status of the deployment objects and the flink job to the extent that the reconciler can work with that information alone. If we do it this way, we can also use the status information that the observer provides to produce other events and aid operations like shutdown which depend on the current deployment status. I think this would satisfy our needs, but I might be missing something that cannot be done if we structure the code this way. I have a PR open <https://github.com/apache/flink-kubernetes-operator/pull/26/commits> which includes some of these proposed changes (as the optional second commit) so that you can easily compare with the current state of the operator. Please let us know what we think! Cheers, Gyula
