[
https://issues.apache.org/jira/browse/SPARK-11058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959320#comment-14959320
]
Lan Jiang commented on SPARK-11058:
-----------------------------------
Sean,
I recreated this problem after I triggered some exceptions in my tasks on
purpose. The resource manager UI reports the final status to be "succeed" but
the job shows up in the "incomplete list" on the spark history server. I do
see the exception thrown by the the driver.
Driver stacktrace:
at
org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1203)
at
org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1192)
at
org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1191)
As to the second possibility, it might be true. I cannot test the scenario on
the existing cluster. I need to launch a new cluster to test it. Will report
back
Lan
> failed spark job reports on YARN as successful
> ----------------------------------------------
>
> Key: SPARK-11058
> URL: https://issues.apache.org/jira/browse/SPARK-11058
> Project: Spark
> Issue Type: Bug
> Components: YARN
> Affects Versions: 1.3.0
> Environment: CDH 5.4
> Reporter: Lan Jiang
> Priority: Minor
>
> I have a spark batch job running on CDH5.4 + Spark 1.3.0. Job is submitted in
> “yarn-client” mode. The job itself failed due to YARN kills several executor
> containers because the containers exceeded the memory limit posed by YARN.
> However, when I went to the YARN resource manager site, it displayed the job
> as successful. I found there was an issue reported in JIRA
> https://issues.apache.org/jira/browse/SPARK-3627, but it says it was fixed in
> Spark 1.2. On Spark history server, it shows the job as “Incomplete”.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
