[help] create HiveMetaStoreClient on windows with problem

2015-06-11 Thread Lee S
Hi all, I installed hive with hcatlog on a linux cluster. And I tried to create HiveMetaStoreClient on windows,but it failes with exception below. Exception in thread main java.lang.NullPointerException at java.lang.ProcessBuilder.start(ProcessBuilder.java:1010) at

Re: Very slow dynamic partition load

2015-06-11 Thread Pradeep Gollakota
Hmm... did your performance increase with the patch you supplied? I do need the partitions in Hive, but I have a separate tool that has the ability to add partitions to the metastore and is definitely much faster than this. I just checked my job again, the actual Hive job completed 24 hours ago

Re: Very slow dynamic partition load

2015-06-11 Thread Pradeep Gollakota
I actually decided to remove one of my 2 partition columns and make it a bucketing column instead... same query completed fully in under 10 minutes with 92 partitions added. This will suffice for me for now. On Thu, Jun 11, 2015 at 2:25 PM, Pradeep Gollakota pradeep...@gmail.com wrote: Hmm...

Re: nested join issue

2015-06-11 Thread Gautam
Thanks for investigating.. Trying to locate the patch that fixes this between 1.1 and 2.0.0-SNAPSHOT. Any leads on what Jira this fix was part of? Or what part of the code the patch is likely to be on? -Gautam. On Thu, Jun 11, 2015 at 8:35 PM, Gopal Vijayaraghavan gop...@apache.org wrote: Hi,

Re: delta file compact take no effect

2015-06-11 Thread Elliot West
What do you see if you issue: SHOW COMPACTIONS; On Thursday, 11 June 2015, r7raul1...@163.com r7raul1...@163.com wrote: I use hive 1.1.0 on hadoop 2.5.0 After I do some update operation on table u_data_txn. My table create many delta file like: drwxr-xr-x - hdfs hive 0 2015-02-06 22:52

Re: Re: delta file compact take no effect

2015-06-11 Thread r7raul1...@163.com
SHOW COMPACTIONS; I can see some info Database Table Partition Type State Worker Start Time default u_data_txn NULL MAJOR initiated NULL 0 Time taken: 0.024 seconds, Fetched: 2 row(s) But after that I still see many delta file. r7raul1...@163.com From: Elliot West Date: 2015-06-11 15:25

Re: Empty Table in MR with union all (created in Tez)

2015-06-11 Thread Gopal Vijayaraghavan
set hive.execution.engine=mr; select * from test_table; --gives empty table Any ideas on why this must be happening? This issue does not occur when I don¹t use a ³union all² query. What is the value of the parameter mapreduce.input.fileinputformat.input.dir.recursive in your

Re: Full text index functionality available?

2015-06-11 Thread 김영우
AFAIK, Hive does not support the built-in features like you mentioned. But using Hive storage handler, you can integrate other components that support data indexing and searching. For instance, there are storage handlers for Solr and ElasticSearch [1][2]. HTH Regards, Youngwoo [1].

nested join issue

2015-06-11 Thread Slava Markeyev
I'm running into a peculiar issue with nested joins and outer select. I see this error on 1.1.0 and 1.2.0 but not 0.13 which seems like a regression. The following query produces no results: select sfrom ( select last.*, action.st2, action.n from ( select purchase.s, purchase.timestamp,

Re: nested join issue

2015-06-11 Thread Gopal Vijayaraghavan
Hi, I'm running into a peculiar issue with nested joins and outer select. I see this error on 1.1.0 and 1.2.0 but not 0.13 which seems like a regression. ... create table events (s string, st2 string, n int, timestamp int); The issue does not seem to be happening in hive-2.0.0-SNAPSHOT, which

RE: Empty Table in MR with union all (created in Tez)

2015-06-11 Thread Gufran Mohammed Pathan
Hi Gopal, My mapreduce.input.fileinputformat.input.dir.recursive was set to false. Setting it to true fixed it. Thanks a lot! Gufran Pathan| +91 7760913355| www.mu-sigma.com | Correlation does not imply causation, but it does waggle its eyebrows suggestively and gesture furtively while

Very slow dynamic partition load

2015-06-11 Thread Pradeep Gollakota
Hi All, I have a table which is partitioned on two columns (customer, date). I'm loading some data into the table using a Hive query. The MapReduce job completed within a few minutes and needs to commit the data to the appropriate partitions. There were about 32000 partitions generated. The