Re: hive on Tez - merging orc files

2015-04-24 Thread patcharee
Hi, I generated the new hive-exec.jar as you suggested. On the sandbox, Hive 0.14 with the new jar file is now using Tez to alter table concate, and It concatenates files correctly on Tez. Thanks! However I also tested on the production cluster using Hive 0.14 as well, merging did not work a

Re: hive on Tez - merging orc files

2015-04-24 Thread Prasanth Jayachandran
You can download the branch-0.14 source code from https://github.com/apache/hive/tree/branch-0.14, apply HIVE-9529-branch-1.0.0.patch from https://issues.apache.org/jira/browse/HIVE-9529 and compile it using “mvn clean install -DskipTests -Phadoop-2,dist”. This will generate tar file under hiv

Re: hive on Tez - merging orc files

2015-04-24 Thread patcharee
Hi, The sandbox 2.2 comes with hive 0.14. Does it also have the bug? If so, how can I patch hive on sandbox? BR, Patcharee On 24. april 2015 09:42, Prasanth Jayachandran wrote: Hi This has been fixed recently https://issues.apache.org/jira/browse/HIVE-9529. Merging is triggered in two diff

Re: hive on Tez - merging orc files

2015-04-24 Thread Prasanth Jayachandran
Hi This has been fixed recently https://issues.apache.org/jira/browse/HIVE-9529. Merging is triggered in two different ways. INSERT/CTAS can trigger merging of small files and CONCATENATE can trigger merging of small files. The later had a bug which generated MR task instead of TEZ task which w

hive on Tez - merging orc files

2015-04-24 Thread patcharee
Hi, Is there anyone using hortonworks sandbox 2.2? I am trying to use hive on Tez on the sandbox. I set the running engine in hive-site.xml to Tez. hive.execution.engine tez Then I ran the script that alters a table to merge small orc files (alter table orc_merge5a part