Remove YuniKorn application CRD should be fine, I don't think we ever reached the GA for this feature, and I don't think anyone is using it. The operator plugin, however, I don't think we should remove, for example the Spark CRD plugin <https://github.com/apache/yunikorn-k8shim/blob/master/pkg/appmgmt/sparkoperator/spark.go> provides the hook to YK in order to update the Spark job status more consistently.
Weiwei On Fri, Jul 28, 2023 at 12:06 PM Craig Condit <ccon...@apache.org> wrote: > I’m definitely in favor of removing the CRD, and sooner rather than later. > It negatively impacts some of the ongoing refactoring tasks as it > influences the recovery pipeline. I think given the alpha state of the CRD, > if there are no objections we can remove this in 1.4.0. > > Craig > > > > On Jul 27, 2023, at 9:41 PM, Wilfred Spiegelenburg <wilfr...@apache.org> > wrote: > > > > We added the YuniKorn application CRD via YUNIKORN-170 [1]. The idea > > was back then that we used the CRD to implement gang scheduling. > > During the later design of gang scheduling we completely stepped away > > from using the CRD as the basis for gang scheduling. Some of the other > > advantages that we were expecting from the CRD were never observed > > (finished state for an application, one point to manage pods). The > > Spark CRD integration was mostly reversed [2] in favour of normal pod > > handling due to issues. > > The second phase for the application CRD YUNIKORN-599 [3] was never > > started due to the limited advantages we expected to get. > > > > There have been no changes in the code or jiras logged against the CRD > > for two years besides making the build work on later K8s versions [4] > > The current usage of the application CRD is limited to the > > TaskGrooupDefinition being used for gang scheduling. > > > > Based on all this I would like to start the discussion on removing the > > application CRD from YuniKorn. Frank Yang has looked at the changes > > needed to remove the CRD and the impact on the code for the K8shim. A > > commit with all the changes can be seen in his repo [5] to reference. > > The change will remove almost 3,000 lines of code just from the K8shim > > repository. There will be some further changes needed to clean up the > > build (K8shim) and helmchart (release). Those changes will be removal > > of scripts and code only. > > > > Before we progress with this further we would like to know: > > * If the application CRD is used by anyone. > > * If it is used, what part(s) of the CRD are used and what is it used > for? > > * Is removing the CRD in YuniKorn 1.4.0 OK or do we need to push to a > > later release. > > > > Objections, comments please let us know. > > > > Thank you, > > Wilfred > > > > [1] https://issues.apache.org/jira/browse/YUNIKORN-170 > > [2] https://issues.apache.org/jira/browse/YUNIKORN-643 > > [3] https://issues.apache.org/jira/browse/YUNIKORN-599 > > [4] > https://github.com/apache/yunikorn-k8shim/commits/master/pkg/apis/yunikorn.apache.org > > [5] > https://github.com/FrankYang0529/yunikorn-k8shim/commit/354248b3c24accd679ff2d5a557c599123a58408 > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org > > For additional commands, e-mail: dev-h...@yunikorn.apache.org > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org > For additional commands, e-mail: dev-h...@yunikorn.apache.org > >
