Once I get hive.execution.engine=spark working, how would I go about
loading portions of my data into memory? Lets say I have a 100TB database
and want to load all of last weeks data in spark memory, is this possible
or even beneficial? Or am I thinking about hive on spark in the wrong way.
I
1) Hive CLI does not talk to HiveServer2
Oh yes. Absolutely. Sorry typo on my end.
2) Beeline talks to HiveServer2 and needs some way to authenticate itself
depending on the configuration of HS2.
HS2 can be configured to authenticate in one of these ways if I'm up to
date:
* NOSASL: no
Have you checked this:
https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started
?
On Wed, Aug 26, 2015 at 11:12 PM, Jeetendra G jeetendr...@housing.com
wrote:
HI All,
I am trying to rum hive on spark means from Hive terminal setting up
execution engine as spark.
I
I am trying to connect hive database(execution.engine value changed to tez)
using Java code... In case of select query its working But in the case
of INSERT getting an error:
The error looks like.
Error while processing statement: FAILED: Execution Error, return code 1
from
I am having a oozie java action and it tries to connect to kerberized Hive
Server 2 with the keytab.
Using keytab to login into kerberos was successful by doing
“UserGroupInformation.loginUserFromKeytab(principalName, KEYTAB);”
But accessing hive server2 stills complains failed to find any
Please refer this wiki page
https://cwiki.apache.org/confluence/display/TEZ/How+to+Diagnose+Tez+App
Best Regard,
Jeff Zhang
From: Sateesh Karuturi
sateesh.karutu...@gmail.commailto:sateesh.karutu...@gmail.com
Reply-To: u...@tez.apache.orgmailto:u...@tez.apache.org
I increase hive.hashtable.initialCapacity to 100 and decrease
hive.hashtable.loadfactor to 0.5 . The query run faster.
r7raul1...@163.com
From: Sergey Shelukhin
Date: 2015-08-28 09:56
To: user
Subject: Re: sql mapjoin very slow
Is the small-side table large, does it have a lot of rows
Is the small-side table large, does it have a lot of rows for the same keys, or
does it have a lot of skew?
Are there lots of misses (where there’d be no value in the small table for the
large table value)?
If you have enough memory you can try increasing initial size and decreasing
load
Are you using MR and Tez? You could try optimized hash table in case of Tez,
although it’s supposed to improve memory, not necessarily perf.
Can you also share characteristics of the query and data? It is surprising to
see so much time for HashMap.get.
From:
I use MR.
My mapjoin config as showed in follow picture:
r7raul1...@163.com
From: Sergey Shelukhin
Date: 2015-08-28 09:21
To: user
Subject: Re: sql mapjoin very slow
Are you using MR and Tez? You could try optimized hash table in case of Tez,
although it’s supposed to improve memory, not
+ hive mail list
Might be a hive bug.
Best Regard,
Jeff Zhang
From: hanked...@emar.commailto:hanked...@emar.com
hanked...@emar.commailto:hanked...@emar.com
Reply-To: user u...@tez.apache.orgmailto:u...@tez.apache.org
Date: Thursday, August 27, 2015 at 5:51 PM
To: user
HI All,
I am trying to rum hive on spark means from Hive terminal setting up
execution engine as spark.
I have copied the hive-default.xml to spark conf directory.
Hive is not able to find the table.giving me error table_name not found?
Can you help me with exact steps how to make spark as a
12 matches
Mail list logo
