[jira] [Comment Edited] (SPARK-36058) Support replicasets/job API
[ https://issues.apache.org/jira/browse/SPARK-36058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17420568#comment-17420568 ] Yikun Jiang edited comment on SPARK-36058 at 9/28/21, 1:11 AM: --- After this, it makes the executor pod allocator pluggable, we could add the VolcanoJobAllocator, and it enables the volcano ability (VolcanoJob with queue, podgoup support...) on excutors side. How about driver side? Do you have any suggestion on it? [~holden] BTW, I just wanna sure that [1][2] abilities targets on driver/excutor side, right? That means we need add PodGroup/Queue abilities on each allocator and driver? [1] scheduler & queue: https://issues.apache.org/jira/browse/SPARK-36059 [2] PodGroup https://issues.apache.org/jira/browse/SPARK-36061 was (Author: yikunkero): After this, it makes the executor pod allocator pluggable, we could add the VolcanoJobAllocator, and it enables the volcano ability (with queue, podgoup and so on...) on excutors side. How about driver side? Do you have any suggestion on it? [~holden] BTW, I just wanna sure that [1][2] abilities targets on driver/excutor side, right? That means we need add PodGroup/Queue abilities on each allocator and driver? [1] scheduler & queue: https://issues.apache.org/jira/browse/SPARK-36059 [2] PodGroup https://issues.apache.org/jira/browse/SPARK-36061 > Support replicasets/job API > --- > > Key: SPARK-36058 > URL: https://issues.apache.org/jira/browse/SPARK-36058 > Project: Spark > Issue Type: Sub-task > Components: Kubernetes >Affects Versions: 3.2.0, 3.3.0 >Reporter: Holden Karau >Assignee: Holden Karau >Priority: Major > Fix For: 3.3.0 > > > Volcano & Yunikorn both support scheduling invidual pods, but they also > support higher level abstractions similar to the vanilla Kube replicasets > which we can use to improve scheduling performance. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-36058) Support replicasets/job API
[ https://issues.apache.org/jira/browse/SPARK-36058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17420568#comment-17420568 ] Yikun Jiang edited comment on SPARK-36058 at 9/27/21, 2:32 PM: --- After this, it makes the executor pod allocator pluggable, we could add the VolcanoJobAllocator, and it enables the volcano ability (with queue, podgoup and so on...) on excutors side. How about driver side? Do you have any suggestion on it? [~holden] BTW, I just wanna sure that [1][2] abilities targets on driver/excutor side, right? That means we need add PodGroup/Queue abilities on each allocator and driver? [1] scheduler & queue: https://issues.apache.org/jira/browse/SPARK-36059 [2] PodGroup https://issues.apache.org/jira/browse/SPARK-36061 was (Author: yikunkero): After this, it makes the executor pod allocator pluggable, we could add the VolcanoJobAllocator, and it enables the volcano ability (with queue, podgoup and so on...) on excutors side. How about driver side? Do you have any suggestion on it? [~holden] BTW, I just wanna sure that [1][2] abilities targets on driver/excutor side, right? That means we need add PodGroup/Queue abilities on each allocator? [1] scheduler & queue: https://issues.apache.org/jira/browse/SPARK-36059 [2] PodGroup https://issues.apache.org/jira/browse/SPARK-36061 > Support replicasets/job API > --- > > Key: SPARK-36058 > URL: https://issues.apache.org/jira/browse/SPARK-36058 > Project: Spark > Issue Type: Sub-task > Components: Kubernetes >Affects Versions: 3.2.0, 3.3.0 >Reporter: Holden Karau >Assignee: Holden Karau >Priority: Major > Fix For: 3.3.0 > > > Volcano & Yunikorn both support scheduling invidual pods, but they also > support higher level abstractions similar to the vanilla Kube replicasets > which we can use to improve scheduling performance. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-36058) Support replicasets/job API
[ https://issues.apache.org/jira/browse/SPARK-36058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17420568#comment-17420568 ] Yikun Jiang edited comment on SPARK-36058 at 9/27/21, 2:31 PM: --- After this, it makes the executor pod allocator pluggable, we could add the VolcanoJobAllocator, and it enables the volcano ability (with queue, podgoup and so on...) on excutors side. How about driver side? Do you have any suggestion on it? [~holden] BTW, I just wanna sure that [1][2] abilities targets on driver/excutor side, right? That means we need add PodGroup/Queue abilities on each allocator? [1] scheduler & queue: https://issues.apache.org/jira/browse/SPARK-36059 [2] PodGroup https://issues.apache.org/jira/browse/SPARK-36061 was (Author: yikunkero): After this, it makes the executor pod allocator pluggable, we could add the VolcanoJobAllocator, and it enables the volcano ability (with queue, podgoup and so on...) on excutors side. How about driver side? Do you have any suggestion on it? [~holden] BTW, I just wanna sure that [1][2] abilities targets on driver/excutor side, right? [1] scheduler & queue: https://issues.apache.org/jira/browse/SPARK-36059 [2] PodGroup https://issues.apache.org/jira/browse/SPARK-36061 > Support replicasets/job API > --- > > Key: SPARK-36058 > URL: https://issues.apache.org/jira/browse/SPARK-36058 > Project: Spark > Issue Type: Sub-task > Components: Kubernetes >Affects Versions: 3.2.0, 3.3.0 >Reporter: Holden Karau >Assignee: Holden Karau >Priority: Major > Fix For: 3.3.0 > > > Volcano & Yunikorn both support scheduling invidual pods, but they also > support higher level abstractions similar to the vanilla Kube replicasets > which we can use to improve scheduling performance. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-36058) Support replicasets/job API
[ https://issues.apache.org/jira/browse/SPARK-36058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17420568#comment-17420568 ] Yikun Jiang edited comment on SPARK-36058 at 9/27/21, 11:22 AM: After this, it makes the executor pod allocator pluggable, we could add the VolcanoJobAllocator, and it enables the volcano ability (with queue, podgoup and so on...) on excutors side. How about driver side? Do you have any suggestion on it? [~holden] BTW, I just wanna sure that [1][2] abilities targets on driver/excutor side, right? [1] scheduler & queue: https://issues.apache.org/jira/browse/SPARK-36059 [2] PodGroup https://issues.apache.org/jira/browse/SPARK-36061 was (Author: yikunkero): After this, it makes the executor pod allocator pluggable, we could add the VolcanoJobAllocator, and it enables the volcano ability (with queue, podgoup and so on...) on excutors side. and looks like[1][2] abilities also targets on excutor side, right? How about driver side? Do you have any suggestion on it? [~holden] [1] scheduler & queue: https://issues.apache.org/jira/browse/SPARK-36059 [2] PodGroup https://issues.apache.org/jira/browse/SPARK-36061 > Support replicasets/job API > --- > > Key: SPARK-36058 > URL: https://issues.apache.org/jira/browse/SPARK-36058 > Project: Spark > Issue Type: Sub-task > Components: Kubernetes >Affects Versions: 3.2.0, 3.3.0 >Reporter: Holden Karau >Assignee: Holden Karau >Priority: Major > Fix For: 3.3.0 > > > Volcano & Yunikorn both support scheduling invidual pods, but they also > support higher level abstractions similar to the vanilla Kube replicasets > which we can use to improve scheduling performance. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
