[ https://issues.apache.org/jira/browse/SPARK-22123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Saisai Shao updated SPARK-22123: -------------------------------- Priority: Minor (was: Major) > Add latest failure reason for task set blacklist > ------------------------------------------------ > > Key: SPARK-22123 > URL: https://issues.apache.org/jira/browse/SPARK-22123 > Project: Spark > Issue Type: Improvement > Components: Scheduler, Spark Core > Affects Versions: 2.1.0, 2.2.0 > Reporter: zhoukang > Assignee: zhoukang > Priority: Minor > Fix For: 2.3.0 > > > Till now , every job which aborted by completed blacklist just show log like > below which has no more information: > {code:java} > Aborting $taskSet because task $indexInTaskSet (partition $partition) cannot > run anywhere due to node and executor blacklist. Blacklisting behavior cannot > run anywhere due to node and executor blacklist.Blacklisting behavior can be > configured via spark.blacklist.*." > {code} > We could add most recent failure reason for taskset blacklist which can be > showed on spark ui to let user know failure reason directly. > An example after modifying: > {code:java} > Aborting TaskSet 0.0 because task 0 (partition 0) cannot run anywhere due to > node and executor blacklist. > Most recent failure: > Some(Lost task 0.1 in stage 0.0 (TID 3,xxx, executor 1): > java.lang.Exception: Fake error! > at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:73) > at org.apache.spark.scheduler.Task.run(Task.scala:99) > at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:305) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > ). > Blacklisting behavior can be configured via spark.blacklist.*. > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org