RE: [DISCUSS] removal of the yunikorn application CRD

[刘达]

it is a pity seeing that application crd will be removed, current, we plan to 
use it as a sole scheduler in our data platform, and i plan to use it for 
scheduling argo workflows, i may modify argo code to use the application crd 
for integrating, even more，i think yunikorn could be the best choice for both 
ai and bigdata usecases, we spend large time on researching volcano and 
yunikorn, and finnaly we choose yunikorn as the scheduler, besides application 
crd, i even expect yunikorn implement volcano's podgroup functions, but until i 
see this issue, i think we will reconsider this scheduler... by the way, what 
is the problem that matters most leading you make such a decision, exept for 
maintaining different verions of k8s...
as i know, using only pod's taskGroups annotation and pod information can't 
really reflect the application status, so a mechanism is needed, either like 
spark operator that build with yunikorn knows diffent kinds of applications, or 
and application crd that external controller can manipulate. yunikorn's queue 
scheduling is fairly advanced than other schedulers in cloud native (for 
examples, default-scheduler, scheduler-plugins), but i just wonder why it is 
not popular than others (at least that's what I see), i think the key reason is 
that yunikorn is not well intergrated with other polular operators and 
frameworks, so i think we should keep the application crd, make it more 
powerful, and pushing more projects to use or intergrate with it.

On 2023/07/28 02:41:18 Wilfred Spiegelenburg wrote:
> We added the YuniKorn application CRD via YUNIKORN-170 [1]. The idea
> was back then that we used the CRD to implement gang scheduling.
> During the later design of gang scheduling we completely stepped away
> from using the CRD as the basis for gang scheduling. Some of the other
> advantages that we were expecting from the CRD were never observed
> (finished state for an application, one point to manage pods). The
> Spark CRD integration was mostly reversed [2] in favour of normal pod
> handling due to issues.
> The second phase for the application CRD YUNIKORN-599 [3] was never
> started due to the limited advantages we expected to get.
> 
> There have been no changes in the code or jiras logged against the CRD
> for two years besides making the build work on later K8s versions [4]
> The current usage of the application CRD is limited to the
> TaskGrooupDefinition being used for gang scheduling.
> 
> Based on all this I would like to start the discussion on removing the
> application CRD from YuniKorn. Frank Yang has looked at the changes
> needed to remove the CRD and the impact on the code for the K8shim. A
> commit with all the changes can be seen in his repo [5] to reference.
> The change will remove almost 3,000 lines of code just from the K8shim
> repository. There will be some further changes needed to clean up the
> build (K8shim) and helmchart (release). Those changes will be removal
> of scripts and code only.
> 
> Before we progress with this further we would like to know:
> * If the application CRD is used by anyone.
> * If it is used, what part(s) of the CRD are used and what is it used for?
> * Is removing the CRD in YuniKorn 1.4.0 OK or do we need to push to a
> later release.
> 
> Objections, comments please let us know.
> 
> Thank you,
> Wilfred
> 
> [1] https://issues.apache.org/jira/browse/YUNIKORN-170
> [2] https://issues.apache.org/jira/browse/YUNIKORN-643
> [3] https://issues.apache.org/jira/browse/YUNIKORN-599
> [4] 
> https://github.com/apache/yunikorn-k8shim/commits/master/pkg/apis/yunikorn.apache.org
> [5] 
> https://github.com/FrankYang0529/yunikorn-k8shim/commit/354248b3c24accd679ff2d5a557c599123a58408
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
> For additional commands, e-mail: dev-h...@yunikorn.apache.org
> 
> 

RE: [DISCUSS] removal of the yunikorn application CRD

[刘达]

it is a pity seeing that application crd will be removed, current, we plan to 
use it as a sole scheduler in our data platform, and i plan to use it for 
scheduling argo workflows, i may modify argo code to use the application crd 
for integrating, even more，i think yunikorn could be the best choice for both 
ai and bigdata usecases, we spend large time on researching volcano and 
yunikorn, and finnaly we choose yunikorn as the scheduler, besides application 
crd, i even expect yunikorn implement volcano's podgroup functions, but until i 
see this issue, i think we will reconsider this scheduler... by the way, what 
is the problem that matters most leading you make such a decision, exept for 
maintaining different verions of k8s...

as i know, using only pod's taskGroups annotation and pod information can't 
really reflect the application status, so a mechanism is needed, either like 
spark operator that build with yunikorn knows diffent kinds of applications, or 
and application crd that external controller can manipulate. yunikorn's queue 
scheduling is fairly advanced than other schedulers in cloud native (for 
examples, default-scheduler, scheduler-plugins), but i just wonder why it is 
not popular than others (at least that's what I see), i think the key reason is 
that yunikorn is not well intergrated with other polular operators and 
frameworks, so i think we should keep the application crd, make it more 
powerful, and pushing more projects to use or intergrate with it.

Re: [DISCUSS] removal of the yunikorn application CRD

[Wilfred Spiegelenburg]

Thank you for the feedback on the removal of the application CRD.
I will go ahead and log a jira and target that for YuniKorn 1.4.

Since Frank Yang has already done the POC for it I will assign the jira to him.

Thank you,
Wilfred

On Sat, 29 Jul 2023 at 16:09, Wilfred Spiegelenburg  wrote:
>
> This is purely about the YuniKorn application CRD. Not about the Spark
> operator integration, we do not want to touch that one
> The Spark link was only to show that a CRD provides limited functionality.
>
> The POC that Frank did was also just the YuniKorn application CRD
>
> Wilfred
>
> On Sat, 29 Jul 2023 at 05:52, Weiwei Yang  wrote:
> >
> > Remove YuniKorn application CRD should be fine, I don't think we ever
> > reached the GA for this feature, and I don't think anyone is using it.
> > The operator plugin, however, I don't think we should remove, for example
> > the Spark CRD plugin
> > 
> > provides
> > the hook to YK in order to update the Spark job status more consistently.
> >
> > Weiwei
> >
> > On Fri, Jul 28, 2023 at 12:06 PM Craig Condit  wrote:
> >
> > > I’m definitely in favor of removing the CRD, and sooner rather than later.
> > > It negatively impacts some of the ongoing refactoring tasks as it
> > > influences the recovery pipeline. I think given the alpha state of the 
> > > CRD,
> > > if there are no objections we can remove this in 1.4.0.
> > >
> > > Craig
> > >
> > >
> > > > On Jul 27, 2023, at 9:41 PM, Wilfred Spiegelenburg 
> > > wrote:
> > > >
> > > > We added the YuniKorn application CRD via YUNIKORN-170 [1]. The idea
> > > > was back then that we used the CRD to implement gang scheduling.
> > > > During the later design of gang scheduling we completely stepped away
> > > > from using the CRD as the basis for gang scheduling. Some of the other
> > > > advantages that we were expecting from the CRD were never observed
> > > > (finished state for an application, one point to manage pods). The
> > > > Spark CRD integration was mostly reversed [2] in favour of normal pod
> > > > handling due to issues.
> > > > The second phase for the application CRD YUNIKORN-599 [3] was never
> > > > started due to the limited advantages we expected to get.
> > > >
> > > > There have been no changes in the code or jiras logged against the CRD
> > > > for two years besides making the build work on later K8s versions [4]
> > > > The current usage of the application CRD is limited to the
> > > > TaskGrooupDefinition being used for gang scheduling.
> > > >
> > > > Based on all this I would like to start the discussion on removing the
> > > > application CRD from YuniKorn. Frank Yang has looked at the changes
> > > > needed to remove the CRD and the impact on the code for the K8shim. A
> > > > commit with all the changes can be seen in his repo [5] to reference.
> > > > The change will remove almost 3,000 lines of code just from the K8shim
> > > > repository. There will be some further changes needed to clean up the
> > > > build (K8shim) and helmchart (release). Those changes will be removal
> > > > of scripts and code only.
> > > >
> > > > Before we progress with this further we would like to know:
> > > > * If the application CRD is used by anyone.
> > > > * If it is used, what part(s) of the CRD are used and what is it used
> > > for?
> > > > * Is removing the CRD in YuniKorn 1.4.0 OK or do we need to push to a
> > > > later release.
> > > >
> > > > Objections, comments please let us know.
> > > >
> > > > Thank you,
> > > > Wilfred
> > > >
> > > > [1] https://issues.apache.org/jira/browse/YUNIKORN-170
> > > > [2] https://issues.apache.org/jira/browse/YUNIKORN-643
> > > > [3] https://issues.apache.org/jira/browse/YUNIKORN-599
> > > > [4]
> > > https://github.com/apache/yunikorn-k8shim/commits/master/pkg/apis/yunikorn.apache.org
> > > > [5]
> > > https://github.com/FrankYang0529/yunikorn-k8shim/commit/354248b3c24accd679ff2d5a557c599123a58408
> > > >
> > > > -
> > > > To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
> > > > For additional commands, e-mail: dev-h...@yunikorn.apache.org
> > > >
> > >
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
> > > For additional commands, e-mail: dev-h...@yunikorn.apache.org
> > >
> > >

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: [DISCUSS] removal of the yunikorn application CRD

[Wilfred Spiegelenburg]

This is purely about the YuniKorn application CRD. Not about the Spark
operator integration, we do not want to touch that one
The Spark link was only to show that a CRD provides limited functionality.

The POC that Frank did was also just the YuniKorn application CRD

Wilfred

On Sat, 29 Jul 2023 at 05:52, Weiwei Yang  wrote:
>
> Remove YuniKorn application CRD should be fine, I don't think we ever
> reached the GA for this feature, and I don't think anyone is using it.
> The operator plugin, however, I don't think we should remove, for example
> the Spark CRD plugin
> 
> provides
> the hook to YK in order to update the Spark job status more consistently.
>
> Weiwei
>
> On Fri, Jul 28, 2023 at 12:06 PM Craig Condit  wrote:
>
> > I’m definitely in favor of removing the CRD, and sooner rather than later.
> > It negatively impacts some of the ongoing refactoring tasks as it
> > influences the recovery pipeline. I think given the alpha state of the CRD,
> > if there are no objections we can remove this in 1.4.0.
> >
> > Craig
> >
> >
> > > On Jul 27, 2023, at 9:41 PM, Wilfred Spiegelenburg 
> > wrote:
> > >
> > > We added the YuniKorn application CRD via YUNIKORN-170 [1]. The idea
> > > was back then that we used the CRD to implement gang scheduling.
> > > During the later design of gang scheduling we completely stepped away
> > > from using the CRD as the basis for gang scheduling. Some of the other
> > > advantages that we were expecting from the CRD were never observed
> > > (finished state for an application, one point to manage pods). The
> > > Spark CRD integration was mostly reversed [2] in favour of normal pod
> > > handling due to issues.
> > > The second phase for the application CRD YUNIKORN-599 [3] was never
> > > started due to the limited advantages we expected to get.
> > >
> > > There have been no changes in the code or jiras logged against the CRD
> > > for two years besides making the build work on later K8s versions [4]
> > > The current usage of the application CRD is limited to the
> > > TaskGrooupDefinition being used for gang scheduling.
> > >
> > > Based on all this I would like to start the discussion on removing the
> > > application CRD from YuniKorn. Frank Yang has looked at the changes
> > > needed to remove the CRD and the impact on the code for the K8shim. A
> > > commit with all the changes can be seen in his repo [5] to reference.
> > > The change will remove almost 3,000 lines of code just from the K8shim
> > > repository. There will be some further changes needed to clean up the
> > > build (K8shim) and helmchart (release). Those changes will be removal
> > > of scripts and code only.
> > >
> > > Before we progress with this further we would like to know:
> > > * If the application CRD is used by anyone.
> > > * If it is used, what part(s) of the CRD are used and what is it used
> > for?
> > > * Is removing the CRD in YuniKorn 1.4.0 OK or do we need to push to a
> > > later release.
> > >
> > > Objections, comments please let us know.
> > >
> > > Thank you,
> > > Wilfred
> > >
> > > [1] https://issues.apache.org/jira/browse/YUNIKORN-170
> > > [2] https://issues.apache.org/jira/browse/YUNIKORN-643
> > > [3] https://issues.apache.org/jira/browse/YUNIKORN-599
> > > [4]
> > https://github.com/apache/yunikorn-k8shim/commits/master/pkg/apis/yunikorn.apache.org
> > > [5]
> > https://github.com/FrankYang0529/yunikorn-k8shim/commit/354248b3c24accd679ff2d5a557c599123a58408
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
> > > For additional commands, e-mail: dev-h...@yunikorn.apache.org
> > >
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
> > For additional commands, e-mail: dev-h...@yunikorn.apache.org
> >
> >

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: [DISCUSS] removal of the yunikorn application CRD

[Weiwei Yang]

Remove YuniKorn application CRD should be fine, I don't think we ever
reached the GA for this feature, and I don't think anyone is using it.
The operator plugin, however, I don't think we should remove, for example
the Spark CRD plugin

provides
the hook to YK in order to update the Spark job status more consistently.

Weiwei

On Fri, Jul 28, 2023 at 12:06 PM Craig Condit  wrote:

> I’m definitely in favor of removing the CRD, and sooner rather than later.
> It negatively impacts some of the ongoing refactoring tasks as it
> influences the recovery pipeline. I think given the alpha state of the CRD,
> if there are no objections we can remove this in 1.4.0.
>
> Craig
>
>
> > On Jul 27, 2023, at 9:41 PM, Wilfred Spiegelenburg 
> wrote:
> >
> > We added the YuniKorn application CRD via YUNIKORN-170 [1]. The idea
> > was back then that we used the CRD to implement gang scheduling.
> > During the later design of gang scheduling we completely stepped away
> > from using the CRD as the basis for gang scheduling. Some of the other
> > advantages that we were expecting from the CRD were never observed
> > (finished state for an application, one point to manage pods). The
> > Spark CRD integration was mostly reversed [2] in favour of normal pod
> > handling due to issues.
> > The second phase for the application CRD YUNIKORN-599 [3] was never
> > started due to the limited advantages we expected to get.
> >
> > There have been no changes in the code or jiras logged against the CRD
> > for two years besides making the build work on later K8s versions [4]
> > The current usage of the application CRD is limited to the
> > TaskGrooupDefinition being used for gang scheduling.
> >
> > Based on all this I would like to start the discussion on removing the
> > application CRD from YuniKorn. Frank Yang has looked at the changes
> > needed to remove the CRD and the impact on the code for the K8shim. A
> > commit with all the changes can be seen in his repo [5] to reference.
> > The change will remove almost 3,000 lines of code just from the K8shim
> > repository. There will be some further changes needed to clean up the
> > build (K8shim) and helmchart (release). Those changes will be removal
> > of scripts and code only.
> >
> > Before we progress with this further we would like to know:
> > * If the application CRD is used by anyone.
> > * If it is used, what part(s) of the CRD are used and what is it used
> for?
> > * Is removing the CRD in YuniKorn 1.4.0 OK or do we need to push to a
> > later release.
> >
> > Objections, comments please let us know.
> >
> > Thank you,
> > Wilfred
> >
> > [1] https://issues.apache.org/jira/browse/YUNIKORN-170
> > [2] https://issues.apache.org/jira/browse/YUNIKORN-643
> > [3] https://issues.apache.org/jira/browse/YUNIKORN-599
> > [4]
> https://github.com/apache/yunikorn-k8shim/commits/master/pkg/apis/yunikorn.apache.org
> > [5]
> https://github.com/FrankYang0529/yunikorn-k8shim/commit/354248b3c24accd679ff2d5a557c599123a58408
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
> > For additional commands, e-mail: dev-h...@yunikorn.apache.org
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
> For additional commands, e-mail: dev-h...@yunikorn.apache.org
>
>

Re: [DISCUSS] removal of the yunikorn application CRD

[Craig Condit]

I’m definitely in favor of removing the CRD, and sooner rather than later. It 
negatively impacts some of the ongoing refactoring tasks as it influences the 
recovery pipeline. I think given the alpha state of the CRD, if there are no 
objections we can remove this in 1.4.0.

Craig


> On Jul 27, 2023, at 9:41 PM, Wilfred Spiegelenburg  
> wrote:
> 
> We added the YuniKorn application CRD via YUNIKORN-170 [1]. The idea
> was back then that we used the CRD to implement gang scheduling.
> During the later design of gang scheduling we completely stepped away
> from using the CRD as the basis for gang scheduling. Some of the other
> advantages that we were expecting from the CRD were never observed
> (finished state for an application, one point to manage pods). The
> Spark CRD integration was mostly reversed [2] in favour of normal pod
> handling due to issues.
> The second phase for the application CRD YUNIKORN-599 [3] was never
> started due to the limited advantages we expected to get.
> 
> There have been no changes in the code or jiras logged against the CRD
> for two years besides making the build work on later K8s versions [4]
> The current usage of the application CRD is limited to the
> TaskGrooupDefinition being used for gang scheduling.
> 
> Based on all this I would like to start the discussion on removing the
> application CRD from YuniKorn. Frank Yang has looked at the changes
> needed to remove the CRD and the impact on the code for the K8shim. A
> commit with all the changes can be seen in his repo [5] to reference.
> The change will remove almost 3,000 lines of code just from the K8shim
> repository. There will be some further changes needed to clean up the
> build (K8shim) and helmchart (release). Those changes will be removal
> of scripts and code only.
> 
> Before we progress with this further we would like to know:
> * If the application CRD is used by anyone.
> * If it is used, what part(s) of the CRD are used and what is it used for?
> * Is removing the CRD in YuniKorn 1.4.0 OK or do we need to push to a
> later release.
> 
> Objections, comments please let us know.
> 
> Thank you,
> Wilfred
> 
> [1] https://issues.apache.org/jira/browse/YUNIKORN-170
> [2] https://issues.apache.org/jira/browse/YUNIKORN-643
> [3] https://issues.apache.org/jira/browse/YUNIKORN-599
> [4] 
> https://github.com/apache/yunikorn-k8shim/commits/master/pkg/apis/yunikorn.apache.org
> [5] 
> https://github.com/FrankYang0529/yunikorn-k8shim/commit/354248b3c24accd679ff2d5a557c599123a58408
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
> For additional commands, e-mail: dev-h...@yunikorn.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org