Re: nested join issue

2015-06-11 Thread Gautam
Thanks for investigating.. Trying to locate the patch that fixes this between 1.1 and 2.0.0-SNAPSHOT. Any leads on what Jira this fix was part of? Or what part of the code the patch is likely to be on? -Gautam. On Thu, Jun 11, 2015 at 8:35 PM, Gopal Vijayaraghavan wrote: > Hi, > > > I'm runnin

Re: nested join issue

2015-06-11 Thread Gopal Vijayaraghavan
Hi, > I'm running into a peculiar issue with nested joins and outer select. I >see this error on 1.1.0 and 1.2.0 but not 0.13 which seems like a >regression. ... > create table events (s string, st2 string, n int, timestamp int); The issue does not seem to be happening in hive-2.0.0-SNAPSHOT, wh

nested join issue

2015-06-11 Thread Slava Markeyev
I'm running into a peculiar issue with nested joins and outer select. I see this error on 1.1.0 and 1.2.0 but not 0.13 which seems like a regression. The following query produces no results: select sfrom ( select last.*, action.st2, action.n from ( select purchase.s, purchase.timestamp, m

Re: Very slow dynamic partition load

2015-06-11 Thread Pradeep Gollakota
I actually decided to remove one of my 2 partition columns and make it a bucketing column instead... same query completed fully in under 10 minutes with 92 partitions added. This will suffice for me for now. On Thu, Jun 11, 2015 at 2:25 PM, Pradeep Gollakota wrote: > Hmm... did your performance

Re: Very slow dynamic partition load

2015-06-11 Thread Pradeep Gollakota
Hmm... did your performance increase with the patch you supplied? I do need the partitions in Hive, but I have a separate tool that has the ability to add partitions to the metastore and is definitely much faster than this. I just checked my job again, the actual Hive job completed 24 hours ago and

Re: Very slow dynamic partition load

2015-06-11 Thread Slava Markeyev
This is something that a few of us have run into. I think the bottleneck is in partition creation calls to the metastore. My work around was HIVE-10385 which optionally removed partition creation in the metastore but this isn't a solution for everyone. If you don't require actual partitions in the

Very slow dynamic partition load

2015-06-11 Thread Pradeep Gollakota
Hi All, I have a table which is partitioned on two columns (customer, date). I'm loading some data into the table using a Hive query. The MapReduce job completed within a few minutes and needs to "commit" the data to the appropriate partitions. There were about 32000 partitions generated. The comm

RE: Empty Table in MR with "union all" (created in Tez)

2015-06-11 Thread Gufran Mohammed Pathan
Hi Gopal, My mapreduce.input.fileinputformat.input.dir.recursive was set to false. Setting it to true fixed it. Thanks a lot! Gufran Pathan| +91 7760913355| www.mu-sigma.com | Correlation does not imply causation, but it does waggle its eyebrows suggestively and gesture furtively while mouthi

Re: switching default execution engine for hiveserver2

2015-06-11 Thread Ayazur Rehman
Thanks a lot !! On Wed, Jun 10, 2015 at 6:28 PM, Hari Subramaniyan < hsubramani...@hortonworks.com> wrote: > Hi ​Ayazur, > > Start hiveserver2 with the parameter as mentioned in the below link > "$HIVE_HOME/bin/hive --service hiveserver2 --hiveconf > hive.execution.engine=tez". > > > https://cwi

[help] create HiveMetaStoreClient on windows with problem

2015-06-11 Thread Lee S
Hi all, I installed hive with hcatlog on a linux cluster. And I tried to create HiveMetaStoreClient on windows,but it failes with exception below. Exception in thread "main" java.lang.NullPointerException at java.lang.ProcessBuilder.start(ProcessBuilder.java:1010) at org.apache.hadoop.util.Shell

Re: Re: delta file compact take no effect

2015-06-11 Thread r7raul1...@163.com
SHOW COMPACTIONS; I can see some info Database Table Partition Type State Worker Start Time default u_data_txn NULL MAJOR initiated NULL 0 Time taken: 0.024 seconds, Fetched: 2 row(s) But after that I still see many delta file. r7raul1...@163.com From: Elliot West Date: 2015-06-11 15:25 To

Re: delta file compact take no effect

2015-06-11 Thread Elliot West
What do you see if you issue: SHOW COMPACTIONS; On Thursday, 11 June 2015, r7raul1...@163.com wrote: > > I use hive 1.1.0 on hadoop 2.5.0 > After I do some update operation on table u_data_txn. > My table create many delta file like: > drwxr-xr-x - hdfs hive 0 2015-02-06 22:52 > /user/hive/ware