Xuefu Zhang created HIVE-15237:
----------------------------------
Summary: Propagate Spark job failure to Hive
Key: HIVE-15237
URL: https://issues.apache.org/jira/browse/HIVE-15237
Project: Hive
Issue Type: Bug
Components: Spark
Affects Versions: 2.1.0
Reporter: Xuefu Zhang
If a Spark job failed for some reason, Hive doesn't get any additional error
message, which makes it very hard for user to figure out why. Here is an
example:
{code}
Status: Running (Hive on Spark job[0])
Job Progress Format
CurrentTime StageId_StageAttemptId:
SucceededTasksCount(+RunningTasksCount-FailedTasksCount)/TotalTasksCount
[StageCost]
2016-11-17 21:32:53,134 Stage-0_0: 0/23 Stage-1_0: 0/28
2016-11-17 21:32:55,156 Stage-0_0: 0(+1)/23 Stage-1_0: 0/28
2016-11-17 21:32:57,167 Stage-0_0: 0(+3)/23 Stage-1_0: 0/28
2016-11-17 21:33:00,216 Stage-0_0: 0(+3)/23 Stage-1_0: 0/28
2016-11-17 21:33:03,251 Stage-0_0: 0(+3)/23 Stage-1_0: 0/28
2016-11-17 21:33:06,286 Stage-0_0: 0(+4)/23 Stage-1_0: 0/28
2016-11-17 21:33:09,308 Stage-0_0: 0(+2,-3)/23 Stage-1_0: 0/28
2016-11-17 21:33:12,332 Stage-0_0: 0(+2,-3)/23 Stage-1_0: 0/28
2016-11-17 21:33:13,338 Stage-0_0: 0(+21,-3)/23 Stage-1_0: 0/28
2016-11-17 21:33:15,349 Stage-0_0: 0(+21,-5)/23 Stage-1_0: 0/28
2016-11-17 21:33:16,358 Stage-0_0: 0(+18,-8)/23 Stage-1_0: 0/28
2016-11-17 21:33:19,373 Stage-0_0: 0(+21,-8)/23 Stage-1_0: 0/28
2016-11-17 21:33:22,400 Stage-0_0: 0(+18,-14)/23 Stage-1_0: 0/28
2016-11-17 21:33:23,404 Stage-0_0: 0(+15,-20)/23 Stage-1_0: 0/28
2016-11-17 21:33:24,408 Stage-0_0: 0(+12,-23)/23 Stage-1_0: 0/28
2016-11-17 21:33:25,417 Stage-0_0: 0(+9,-26)/23 Stage-1_0: 0/28
2016-11-17 21:33:26,420 Stage-0_0: 0(+12,-26)/23 Stage-1_0: 0/28
2016-11-17 21:33:28,427 Stage-0_0: 0(+9,-29)/23 Stage-1_0: 0/28
2016-11-17 21:33:29,432 Stage-0_0: 0(+12,-29)/23 Stage-1_0: 0/28
2016-11-17 21:33:31,444 Stage-0_0: 0(+18,-29)/23 Stage-1_0: 0/28
2016-11-17 21:33:34,464 Stage-0_0: 0(+18,-29)/23 Stage-1_0: 0/28
Status: Failed
FAILED: Execution Error, return code 3 from
org.apache.hadoop.hive.ql.exec.spark.SparkTask
{code}
It would be better if we can propagate Spark error to Hive.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
