Creating Surrogate Keys in Hive

2017-11-21 Thread kishore kumar
Hi, Could some one suggest how to create surrogate keys sequentially in hive ? -- Thanks, Kishore.

Re: Error "Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient"

2017-11-21 Thread Furcy Pin
Hi, I recommend you try using HiveServer2 and beeline as explained here: https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-RunningHiveServer2andBeeline.1 In particular, please make sure you have run "bin/schematool -dbType derby -initSchema" and that it returned succe

Re: Creating Surrogate Keys in Hive

2017-11-21 Thread Alan Gates
It isn't possible to guarantee sequential keys because tasks run in parallel. You can write a UDF to assign a unique id or sequential ids within a task. Alan. On Tue, Nov 21, 2017 at 3:53 AM, kishore kumar wrote: > Hi, > > Could some one suggest how to create surrogate keys sequentially in hiv

Hive +Tez+LLAP does not have obvious performance improvement than HIVE + Tez

2017-11-21 Thread Jia, Ke A
Hi all, Now, we are running the benchmark of Hive +Tez+LLAP and Apache Tez in TPC-DS with 3TB orc data. But the result of Hive +Tez+LLAP is almost similar with hive+Tez and some queries may be poorer than hive+tez . The following is our cluster and llap configuration, Cluster: 1 master + 7 slave

Re: Hive +Tez+LLAP does not have obvious performance improvement than HIVE + Tez

2017-11-21 Thread Gopal Vijayaraghavan
Hi, > Please help us find whether we use the wrong configuration. Thanks for your > help. Since there are no details, I'm not sure what configuration you are discussing here. A first step would be to check if LLAP cache is actually being used (the LLAP IO in the explain), vectorization is be

RE: Hive +Tez+LLAP does not have obvious performance improvement than HIVE + Tez

2017-11-21 Thread Jia, Ke A
Hi Gopal, Thanks for your reply. > A first step would be to check if LLAP cache is actually being used (the LLAP > IO in the explain), vectorization is being used (llap, vectorized for tasks), > that the column stats show as COMPLETE (instead of NONE). 1. For the LLAP cache, we have enable the LL

migrate hive cli to beeline

2017-11-21 Thread 游垂凯
Hello everyone: Recently,I want to migrate hive cli to beeline,but i'm reffered to cloudera official site about hive beeline(https://www.cloudera.com/documentation/enterprise/5-8-x/topics/cdh_ig_hive_install.html#concept_alp_4kl_3q), it describe that the beeline cli should use a heap siz