Chao created HIVE-9455:
--------------------------
Summary: MapJoin task shouldn't start if HashTableSink task failed
[Spark Branch]
Key: HIVE-9455
URL: https://issues.apache.org/jira/browse/HIVE-9455
Project: Hive
Issue Type: Sub-task
Affects Versions: spark-branch
Reporter: Chao
While playing with {{auto_join25.q}}, I noticed that even though the task for
hash table sink failed, HOS will still continue launch the task for map join.
This is not the desired result. Instead, like MR, we should abandon the second
task.
Console output:
{code}
Total jobs = 2
Launching Job 1 out of 2
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapreduce.job.reduces=<number>
Query Hive on Spark job[0] stages:
0
Status: Running (Hive on Spark job[0])
Job Progress Format
CurrentTime StageId_StageAttemptId:
SucceededTasksCount(+RunningTasksCount-FailedTasksCount)/TotalTasksCount
[StageCost]
2015-01-23 16:18:14,604 Stage-0_0: 0/1
2015-01-23 04:18:14 Processing rows: 4 Hashtable size: 3
Memory usage: 119199408 percentage: 0.25
2015-01-23 16:18:15,611 Stage-0_0: 0(+0,-1)/1
Status: Finished successfully in 1.07 seconds
Launching Job 2 out of 2
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapreduce.job.reduces=<number>
2015-01-23 16:22:27,854 Stage-1_0: 0(+0,-1)/1
Status: Finished successfully in 1.01 seconds
Loading data to table default.dest1
Table default.dest1 stats: [numFiles=0, numRows=0, totalSize=0, rawDataSize=0]
OK
Time taken: 311.979 seconds
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
