Hive counters for records read and written

2015-06-16 Thread Hemanth Meka
Hi, I can see that two new counters have been added for hive (RECORDS_IN/RECORDS_OUT) in hive 0.14. Prior to this release which counters could be used to get the records read by hive job and records written. Because i noticed that in hive 0.14 for a few hive jobs i see map_input_records

Re: collecting lists in non-natural order / SORT BY on columns which are not part of the final result set

2015-06-16 Thread Michael Häusler
Hi Robin, thanks a lot for the hint about the WITH statement. This is comparable to using a subselect. E.g., I could rewrite my query from: SELECT actor, COLLECT_LIST(action) AS actions FROM foobar GROUP BY actor SORT BY actor, ts; to: SELECT actor,

Re: Alter table statement for create table like-tables

2015-06-16 Thread Jason Dere
Probably has to do with the fact that it is an Avro table. I don't have any experience using Avro, but maybe take a look at https://issues.apache.org/jira/browse/HIVE-7446 for some of the issues described there, or maybe look at the test that was added for that Jira. On Jun 16, 2015, at 2:42

error on hive insert query

2015-06-16 Thread Sateesh Karuturi
iam using *hive 1.0.0* and *tez 0.5.2.* when i set *hive.execution.engine* value in hive-site.xml to *tez*select query works well... but in case of *insert* getting error. the query is : *insert into table tablename values(intvalue,'string value');* and the error is : *FAILED: Execution Error,

Re: Merging small files in partitions

2015-06-16 Thread Edward Capriolo
https://github.com/edwardcapriolo/filecrush On Tue, Jun 16, 2015 at 5:05 PM, Chagarlamudi, Prasanth prasanth.chagarlam...@epsilon.com wrote: Hello, I am looking for an optimized way to merge small files in hive partitions into one big file. I came across *Alter Table/Partition

Re: error on hive insert query

2015-06-16 Thread Hitesh Shah
Unless you can pinpoint the problem to something Tez specific, hive-specific questions might be better off being asked on user@hive initially as there is a larger group there that understands Hive as compared to the Tez community. FWIW, TezTask error 1 means “something in the Hive layer using

Merging small files in partitions

2015-06-16 Thread Chagarlamudi, Prasanth
Hello, I am looking for an optimized way to merge small files in hive partitions into one big file. I came across Alter Table/Partition Concatenate https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-AlterTable/PartitionConcatenate. Doc says this only works for

Re: error on hive insert query

2015-06-16 Thread Jianfeng (Jeff) Zhang
You can refer to the following link to figure out whether this is tez problem https://cwiki.apache.org/confluence/display/TEZ/How+to+Diagnose+Tez+App Best Regard, Jeff Zhang From: Sateesh Karuturi sateesh.karutu...@gmail.commailto:sateesh.karutu...@gmail.com Reply-To:

hive cbo calciteplanner

2015-06-16 Thread wangzhenhua (G)
Hi all, I'm reading the source code of Hive cbo (CalcaitePlanner), but I find it hard to follow. Listed below are some of the questions: 1. What's the relationship between HepPlanner and HiveVolcanoPlanner? 2. I don't have a clue about these concepts: clusters, traitDef and collectGarbage().

Re: Re: delta file compact take no effect

2015-06-16 Thread r7raul1...@163.com
Any help? r7raul1...@163.com From: r7raul1...@163.com Date: 2015-06-11 15:53 To: user Subject: Re: Re: delta file compact take no effect SHOW COMPACTIONS; I can see some info Database Table Partition Type State Worker Start Time default u_data_txn NULL MAJOR initiated NULL 0 Time taken:

Re: delta file compact take no effect

2015-06-16 Thread Alan Gates
Is the config you give on your metastore or your client? The worker thread and initiator must be started on the metastore. Alan. r7raul1...@163.com mailto:r7raul1...@163.com June 16, 2015 at 22:38 Any help?

Left function

2015-06-16 Thread Ravisankar Mani
Hi every one, how to get leftmost length of characters from the string in hive? In Mysql or sq has specific function LEFT(string,length) Could you please help any other way to achieve this scenario? Regards Ravisnkar

prepareStatement.getMetaData problem

2015-06-16 Thread zhangjp
Hi, can anyone to fix this bug? I think it's important https://issues.apache.org/jira/browse/HIVE-6992 is there any other way to getMetaData before run the sql? thanks!

Re: Alter table statement for create table like-tables

2015-06-16 Thread Julian Keppel
*Push* Does no one have an idea or hit similar issues? 2015-06-09 15:46 GMT+02:00 Julian Keppel juliankeppel1...@gmail.com: I use Hive Version 1.1.0 in Cloudera CDH 5.4.0. I have created an external table: CREATE EXTERNAL TABLE *tableA* ROW FORMAT SERDE

Re: Left function

2015-06-16 Thread Nitin Pawar
try using substr function On Tue, Jun 16, 2015 at 3:03 PM, Ravisankar Mani rrav...@gmail.com wrote: Hi every one, how to get leftmost length of characters from the string in hive? In Mysql or sq has specific function LEFT(string,length) Could you please help any other way to achieve