Re: [DISCUSS] Operators and applications

2021-03-30 Thread Julia Kinga Marton
Hi! In my opinion this is the right way as well. Also the App CRD is already following this pattern: it`s controller is handling the CRD related events, and every Pod related event is handled by the general operator. Related to the status of an application: right now, the source of truth is the

Re: [DISCUSS] Operators and applications

2021-03-29 Thread Chaoran Yu
Thanks Wilfred for the proposal. I agree with the overall approach. To summarize the desired responsibilities of a generic 3rd party app management plugin (only the Spark operator plugin for now), combining with what Weiwei said, * It will only react to lifecycle events (Add, Update, Delete etc)

Re: [DISCUSS] Operators and applications

2021-03-29 Thread Weiwei Yang
Hi Wilfred The original idea was to have each app mgmt plugin, e.g spark operator plugin, manage the certain type of app's lifecycle independently. That means each pod on K8s will only be seen and monitored by one app mgmt plugin. The problems we found earlier were because it goes against this

[DISCUSS] Operators and applications

2021-03-29 Thread Wilfred Spiegelenburg
Hi, Based on testing that was performed around gang scheduling and the spark operator by Bowen Li and Chaoran Yu we found that the behaviour around the operator was far from optimal. YUNIKORN-558 was logged to help with the integration. We did