[GitHub] [spark] pan3793 opened a new pull request, #38732: [SPARK-41210][K8S] Window based executor failure tracking mechanism
pan3793 opened a new pull request, #38732: URL: https://github.com/apache/spark/pull/38732 ### What changes were proposed in this pull request? Fail Spark Application when number of executor failures reach threshold. ### Why are the changes needed? Sometimes, the executors can not launch successfully because of the wrong configuration, but in K8s, Driver does not know that, and just keep requesting new executors. This adds functionality similar to YARN[1] to K8s. [1] [SPARK-6735](https://issues.apache.org/jira/browse/SPARK-6735) ### Does this PR introduce _any_ user-facing change? New feature. ### How was this patch tested? New UT added, and manually tested in internal K8s cluster. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] pan3793 opened a new pull request, #38732: [SPARK-41210][K8S] Window based executor failure tracking mechanism
pan3793 opened a new pull request, #38732: URL: https://github.com/apache/spark/pull/38732 ### What changes were proposed in this pull request? Fail Spark Application when executor failures reach threshold. ### Why are the changes needed? Sometimes, executor can not launch successful because of wrong configuration, but in K8s, Driver does not know that, just keep requesting new executor. This adds functionality similar to YARN[1][2] to K8s. [1] [SPARK-7451](https://issues.apache.org/jira/browse/SPARK-7451) [2] [SPARK-6735](https://issues.apache.org/jira/browse/SPARK-6735) ### Does this PR introduce _any_ user-facing change? New feature. ### How was this patch tested? UT will be added soon -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
