Hello all After some digging and remote java debug, I found out that in file HiveServer2.java line 1001:
if (policyContainer.size() > 0 ) {
zKClientForPrivSync = startZookeeperClient(hiveConf) ;
String rootNamespace = hiveConf.getVar(HiveConf.ConfVars.
HIVE_SERVER2_ZOOKEEPER_NAMESPACE ) ;
In this case policyContainer.size is > 1 has there's an object
RangerHiveAuthorizer
In this case I'm a bit puzzled, why hive would need zookeeper has Ranger is
supposed to maintains the policy the same for all node ?
Have a very nice day.
De: "Antoine DUBOIS" <antoine.dub...@cc.in2p3.fr>
À: user@hive.apache.org
Envoyé: Lundi 6 Septembre 2021 10:03:59
Objet: Re: Question regarding lock manager
Hello Alan,
Thank you for your answer,
I'm pretty sure I've respected the guide provided and did not set any discovery
service.
However I'm trying to setup ranger as well and though I didn't though it could
cause the issue, after futher investigation I suspect ranger might be causing
the issue.
Hadoop 3.1.4
Hive 3.1.2
Ranger 2.1.0
Tez 0.9.2
It might come from imcompatibles version but I cannot find a proper
compatibility matrix for all hadoop ecosystem.
Thank you very much.
Antoine
De: "Alan Gates" <alanfga...@gmail.com>
À: user@hive.apache.org
Envoyé: Vendredi 3 Septembre 2021 17:51:41
Objet: Re: Question regarding lock manager
You do not need ZooKeeper to use ACID in Hive. The first thing I would check is
that you have configured your system as described on this page: [
https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions |
https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions ] . Also,
make sure you have not set hive.lock.manager to zookeeper.
There are other features in Hive that can optionally use ZK, such as the
discovery service for HiveServer2, you'll want to make sure this isn't coming
from there rather than from the transaction system.
Alan.
On Thu, Sep 2, 2021 at 7:46 AM Antoine DUBOIS < [
mailto:antoine.dub...@cc.in2p3.fr | antoine.dub...@cc.in2p3.fr ] > wrote:
Hello,
I'm trying to configure ACID hive in a kerberos environment with :
Hadoop 3.1.4 deployed in HA considered working
and now I'm trying to setup hive with remote metastore and ACID configuration.
I may misunderstand what is written in documentation [
https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions |
https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions ] , but to
me it's cleary stated that DbTxnManager do not need zookeeper to run properly,
However it seems it in fact needs a zookeeper running somewhere as I have
several attempt to connect to a local zookeeper instance:
2021-09 -02T 15:22:21,708 INFO [main-SendThread(localhost: 2181 )]
client.ZooKeeperSaslClient : Client will use GSSAPI as SASL mechanism.
2021-09-02T15:22:21,708 DEBUG [main-SendThread(localhost: 2181 )]
client.ZooKeeperSaslClient : creating sasl client: client=h*******
;service=zookeeper;serviceHostname=localhost
2021-09-02T15:22:21,709 INFO [main-SendThread(localhost: 2181 )]
zookeeper.ClientCnxn : Opening socket connection to server localhost/ 127 . 0 .
0 . 1 : 2181 . Will attempt to SASL-authenticate using Login Context section
'HiveZooKeeperClient'
2021-09-02T15:22:21,710 WARN [main-SendThread(localhost: 2181 )]
zookeeper.ClientCnxn : Session 0x0 for server null , unexpected error, closing
socket connection and attempting reconnect
java.net.ConnectException : Connection refused
Could you please help me to understand properly the documentation ? Do I need
or not a zookeeper instance if using DbTxnManager and if so, why it isn't
stated precisely and explicitely in the documentation ?
I hope you have the best of day.
Antoine DUBOIS
smime.p7s
Description: S/MIME Cryptographic Signature
