Re: merge small orc files

Hi Gopal, Thanks for your explanation. What could be the case that SET hive.merge.orcfile.stripe.level=true alter table table concatenate do not work? I have a dynamic partitioned table (stored as orc). I tried to alter concatenate, but it did not work. See my test result. hive SET

mapred.reduce.tasks

In MapReduce job how reduce tasks numbers are decided ? I haven't override the mapred.reduce.tasks property and its creating ~700 reduce tasks. Thanks

Re: merge small orc files

Hi Gopal, The table created is not a bucketed table, but a dynamic partitioned table. I took the script test from https://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/orc_merge7.q - create table orc_merge5 (userid bigint, string1 string, subtype double, decimal1

MapredContext not available when tez enabled

We have a UDF to collect some counts during Hive execution. It has been working fine until tez is enabled. A bit digging shows that GenericUDF#configure method was not called. So in this case, is it possible to get counters through other means, or we have to implement Counter concept

Re: MapredContext not available when tez enabled

A bit digging shows that GenericUDF#configure method was not called. So in this case, is it possible to get counters through other means, or we have to implement Counter concept ourselves? You should be getting a TezContext object there (which inherits from MapRedContext). And the method

RE: mapred.reduce.tasks

Hi In MapReduce , number of reducers launched is set by property “mapreduce.job.reduces”. And via java API Job#setNumReduceTasks(int tasks). In your MR job, somewhere in the program they are setting number of reducer task using java API or property. May be you can check the MR job or property

RE: UDF cannot be found when the query is submitted via templeton

What we do is to upload the script first into HDFS, then use the file option in WebHCat to submit the queries - so I think the REST call should not matter... right? Xiaoyong From: Eugene Koifman [mailto:ekoif...@hortonworks.com] Sent: Monday, April 20, 2015 10:32 PM To: user@hive.apache.org

Re: merge small orc files

alter table table concatenate do not work? I have a dynamic partitioned table (stored as orc). I tried to alter concatenate, but it did not work. See my test result. ORC fast concatenate does work on partitioned tables, but it doesn¹t work on bucketed tables. Bucketed tables cannot merge