[
https://issues.apache.org/jira/browse/HIVE-15237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-15237:
--------------------------
Attachment: HIVE-15237.2.patch
Thanks [~xuefuz] for the patch. I made some modifications based on your work.
Tried to parse the root cause from the Throwable. Then print the root cause to
console and leave the detailed message to the log.
Please try it and let me know your opinions. Thanks.
> Propagate Spark job failure to Hive
> -----------------------------------
>
> Key: HIVE-15237
> URL: https://issues.apache.org/jira/browse/HIVE-15237
> Project: Hive
> Issue Type: Bug
> Components: Spark
> Affects Versions: 2.1.0
> Reporter: Xuefu Zhang
> Assignee: Rui Li
> Attachments: HIVE-15237.2.patch, HIVE-15237.patch
>
>
> If a Spark job failed for some reason, Hive doesn't get any additional error
> message, which makes it very hard for user to figure out why. Here is an
> example:
> {code}
> Status: Running (Hive on Spark job[0])
> Job Progress Format
> CurrentTime StageId_StageAttemptId:
> SucceededTasksCount(+RunningTasksCount-FailedTasksCount)/TotalTasksCount
> [StageCost]
> 2016-11-17 21:32:53,134 Stage-0_0: 0/23 Stage-1_0: 0/28
> 2016-11-17 21:32:55,156 Stage-0_0: 0(+1)/23 Stage-1_0: 0/28
> 2016-11-17 21:32:57,167 Stage-0_0: 0(+3)/23 Stage-1_0: 0/28
> 2016-11-17 21:33:00,216 Stage-0_0: 0(+3)/23 Stage-1_0: 0/28
> 2016-11-17 21:33:03,251 Stage-0_0: 0(+3)/23 Stage-1_0: 0/28
> 2016-11-17 21:33:06,286 Stage-0_0: 0(+4)/23 Stage-1_0: 0/28
> 2016-11-17 21:33:09,308 Stage-0_0: 0(+2,-3)/23 Stage-1_0: 0/28
> 2016-11-17 21:33:12,332 Stage-0_0: 0(+2,-3)/23 Stage-1_0: 0/28
> 2016-11-17 21:33:13,338 Stage-0_0: 0(+21,-3)/23 Stage-1_0: 0/28
> 2016-11-17 21:33:15,349 Stage-0_0: 0(+21,-5)/23 Stage-1_0: 0/28
> 2016-11-17 21:33:16,358 Stage-0_0: 0(+18,-8)/23 Stage-1_0: 0/28
> 2016-11-17 21:33:19,373 Stage-0_0: 0(+21,-8)/23 Stage-1_0: 0/28
> 2016-11-17 21:33:22,400 Stage-0_0: 0(+18,-14)/23 Stage-1_0: 0/28
> 2016-11-17 21:33:23,404 Stage-0_0: 0(+15,-20)/23 Stage-1_0: 0/28
> 2016-11-17 21:33:24,408 Stage-0_0: 0(+12,-23)/23 Stage-1_0: 0/28
> 2016-11-17 21:33:25,417 Stage-0_0: 0(+9,-26)/23 Stage-1_0: 0/28
> 2016-11-17 21:33:26,420 Stage-0_0: 0(+12,-26)/23 Stage-1_0: 0/28
> 2016-11-17 21:33:28,427 Stage-0_0: 0(+9,-29)/23 Stage-1_0: 0/28
> 2016-11-17 21:33:29,432 Stage-0_0: 0(+12,-29)/23 Stage-1_0: 0/28
> 2016-11-17 21:33:31,444 Stage-0_0: 0(+18,-29)/23 Stage-1_0: 0/28
> 2016-11-17 21:33:34,464 Stage-0_0: 0(+18,-29)/23 Stage-1_0: 0/28
> Status: Failed
> FAILED: Execution Error, return code 3 from
> org.apache.hadoop.hive.ql.exec.spark.SparkTask
> {code}
> It would be better if we can propagate Spark error to Hive.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
