[jira] [Commented] (SPARK-6735) Provide options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it.
[ https://issues.apache.org/jira/browse/SPARK-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15049958#comment-15049958 ] Saisai Shao commented on SPARK-6735: I've submitted a patch to continue this work (https://github.com/apache/spark/pull/10241), please help to review, thanks a lot. > Provide options to make maximum executor failure count ( which kills the > application ) relative to a window duration or disable it. > --- > > Key: SPARK-6735 > URL: https://issues.apache.org/jira/browse/SPARK-6735 > Project: Spark > Issue Type: Improvement > Components: Spark Submit, YARN >Affects Versions: 1.2.0, 1.2.1, 1.3.0 >Reporter: Twinkle Sachdeva > > Currently there is a setting (spark.yarn.max.executor.failures ) which tells > maximum number of executor failures, after which Application fails. > For long running applications, user can require not to kill the application > at all or will require such setting relative to a window duration. This > improvement is ti provide such options to make maximum executor failure count > ( which kills the application ) relative to a window duration or disable it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6735) Provide options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it.
[ https://issues.apache.org/jira/browse/SPARK-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15049950#comment-15049950 ] Apache Spark commented on SPARK-6735: - User 'jerryshao' has created a pull request for this issue: https://github.com/apache/spark/pull/10241 > Provide options to make maximum executor failure count ( which kills the > application ) relative to a window duration or disable it. > --- > > Key: SPARK-6735 > URL: https://issues.apache.org/jira/browse/SPARK-6735 > Project: Spark > Issue Type: Improvement > Components: Spark Submit, YARN >Affects Versions: 1.2.0, 1.2.1, 1.3.0 >Reporter: Twinkle Sachdeva > > Currently there is a setting (spark.yarn.max.executor.failures ) which tells > maximum number of executor failures, after which Application fails. > For long running applications, user can require not to kill the application > at all or will require such setting relative to a window duration. This > improvement is ti provide such options to make maximum executor failure count > ( which kills the application ) relative to a window duration or disable it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6735) Provide options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it.
[ https://issues.apache.org/jira/browse/SPARK-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14608280#comment-14608280 ] Thomas Graves commented on SPARK-6735: -- Pull request was up but didn't have time to do rework for some comments if someone else wants to take this over. https://github.com/apache/spark/pull/5449 Provide options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it. --- Key: SPARK-6735 URL: https://issues.apache.org/jira/browse/SPARK-6735 Project: Spark Issue Type: Improvement Components: Spark Submit, YARN Affects Versions: 1.2.0, 1.2.1, 1.3.0 Reporter: Twinkle Sachdeva Currently there is a setting (spark.yarn.max.executor.failures ) which tells maximum number of executor failures, after which Application fails. For long running applications, user can require not to kill the application at all or will require such setting relative to a window duration. This improvement is ti provide such options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6735) Provide options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it.
[ https://issues.apache.org/jira/browse/SPARK-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487670#comment-14487670 ] Sandy Ryza commented on SPARK-6735: --- Hi [~twinkle], can you submit the PR against the main Spark project. Provide options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it. --- Key: SPARK-6735 URL: https://issues.apache.org/jira/browse/SPARK-6735 Project: Spark Issue Type: Improvement Components: Spark Submit, YARN Affects Versions: 1.2.0, 1.2.1, 1.3.0 Reporter: Twinkle Sachdeva Currently there is a setting (spark.yarn.max.executor.failures ) which tells maximum number of executor failures, after which Application fails. For long running applications, user can require not to kill the application at all or will require such setting relative to a window duration. This improvement is ti provide such options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6735) Provide options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it.
[ https://issues.apache.org/jira/browse/SPARK-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14483018#comment-14483018 ] Twinkle Sachdeva commented on SPARK-6735: - Created a PR here : https://github.com/twinkle-sachdeva/spark/pull/1 Provide options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it. --- Key: SPARK-6735 URL: https://issues.apache.org/jira/browse/SPARK-6735 Project: Spark Issue Type: Improvement Components: Spark Submit, YARN Affects Versions: 1.2.0, 1.2.1, 1.3.0 Reporter: Twinkle Sachdeva Currently there is a setting (spark.yarn.max.executor.failures ) which tells maximum number of executor failures, after which Application fails. For long running applications, user can require not to kill the application at all or will require such setting relative to a window duration. This improvement is ti provide such options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
