Amit Pathak created HIVE-14586:
----------------------------------
Summary: Hive : with multi insert query: FAILED: SemanticException
Should not happened
Key: HIVE-14586
URL: https://issues.apache.org/jira/browse/HIVE-14586
Project: Hive
Issue Type: Bug
Reporter: Amit Pathak
I am using multi insert query for optimization purpose, surely it helps me a
lot but with each day run, I can find 3 to 4 id's (having count more than 10
Million) taking too much time at reducer. to fix this I have implemented
skewjoin optimization properties but it's throwing
"FAILED: SemanticException Should not happened"
Properties which I am using
set hive.exec.compress.output=true;
set mapreduce.output.fileoutputformat.compress=true;
set
mapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.SnappyCodec;
set mapreduce.output.fileoutputformat.compress.type=BLOCK;
SET hive.optimize.skewjoin=true;
set hive.exec.compress.intermediate=true;
set
hive.intermediate.compression.codec=org.apache.hadoop.io.compress.SnappyCodec;
set hive.intermediate.compression.type=BLOCK;
set hive.optimize.skewjoin=true;
set hive.skewjoin.key=100000;
set hive.skewjoin.mapjoin.map.tasks=10000;
set hive.skewjoin.mapjoin.min.split=33554432;
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
