[jira] [Created] (SPARK-26717) Support PodPriority for spark driver and executor on kubernetes
Li Gao created SPARK-26717: -- Summary: Support PodPriority for spark driver and executor on kubernetes Key: SPARK-26717 URL: https://issues.apache.org/jira/browse/SPARK-26717 Project: Spark Issue Type: Wish Components: Kubernetes Affects Versions: 2.4.0 Reporter: Li Gao Hi, I'd like to see whether we could enhance the current Spark 2.4 on k8s support to bring in the `PodPriority` feature thats available since k8s v1.11+ [https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/] This is helpful in a shared cluster environment to ensure SLA for jobs with different priorities. Thanks! Li -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25515) Add a config property for disabling auto deletion of PODS for debugging.
[ https://issues.apache.org/jira/browse/SPARK-25515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16645217#comment-16645217 ] Li Gao commented on SPARK-25515: I'd like to create a PR for this to enable such option. > Add a config property for disabling auto deletion of PODS for debugging. > > > Key: SPARK-25515 > URL: https://issues.apache.org/jira/browse/SPARK-25515 > Project: Spark > Issue Type: Improvement > Components: Kubernetes >Affects Versions: 3.0.0 >Reporter: Prashant Sharma >Priority: Major > > Currently, if a pod fails to start due to some failure, it gets removed and > new one is attempted. These sequence of events go on, until the app is > killed. Given the speed of creation and deletion, it becomes difficult to > debug the reason for failure. > So adding a configuration parameter to disable auto-deletion of pods, will be > helpful for debugging. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-25515) Add a config property for disabling auto deletion of PODS for debugging.
[ https://issues.apache.org/jira/browse/SPARK-25515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16645217#comment-16645217 ] Li Gao edited comment on SPARK-25515 at 10/10/18 4:28 PM: -- I'd like to create a PR for this to enable such option as we have similar needs. was (Author: ligao...@gmail.com): I'd like to create a PR for this to enable such option. > Add a config property for disabling auto deletion of PODS for debugging. > > > Key: SPARK-25515 > URL: https://issues.apache.org/jira/browse/SPARK-25515 > Project: Spark > Issue Type: Improvement > Components: Kubernetes >Affects Versions: 3.0.0 >Reporter: Prashant Sharma >Priority: Major > > Currently, if a pod fails to start due to some failure, it gets removed and > new one is attempted. These sequence of events go on, until the app is > killed. Given the speed of creation and deletion, it becomes difficult to > debug the reason for failure. > So adding a configuration parameter to disable auto-deletion of pods, will be > helpful for debugging. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
