Hugo WARIN created RANGER-4950:
----------------------------------
Summary: Hive-plugin not working with Hadoop 3.3.6 and Hive 3.1.3
Key: RANGER-4950
URL: https://issues.apache.org/jira/browse/RANGER-4950
Project: Ranger
Issue Type: Bug
Components: plugins
Affects Versions: 2.5.0
Reporter: Hugo WARIN
Hello,
I'm working on an hadoop 3.3.6 stack with Hive 3.1.3, tez 0.10.2 and spark
3.5.1.
The issue is ranger cannot connect to my hive service. I have those errors :
hiveserver2.out :
{code:java}
12:16:43.952 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.plugin.classloader.RangerPluginClassLoader - <==
RangerPluginClassLoader.findResources(META-INF/services/javax.ws.rs.ext.MessageBodyWriter)
12:16:43.957 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.plugin.classloader.RangerPluginClassLoader - ==>
RangerPluginClassLoader.loadClass(javax.mail.MessagingException)
12:16:43.957 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.plugin.classloader.RangerPluginClassLoader -
RangerPluginClassLoader.loadClass(javax.mail.MessagingException): calling
childClassLoader.findClass()
12:16:43.957 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.plugin.classloader.RangerPluginClassLoader - ==>
RangerPluginClassLoader.findClass(javax.mail.MessagingException)
12:16:43.957 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.plugin.classloader.RangerPluginClassLoader -
RangerPluginClassLoader.findClass(javax.mail.MessagingException): calling
childClassLoader().findClass()
12:16:43.957 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.plugin.classloader.RangerPluginClassLoader -
RangerPluginClassLoader.findClass(javax.mail.MessagingException): calling
componentClassLoader.findClass()
12:16:43.957 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.plugin.classloader.RangerPluginClassLoader -
RangerPluginClassLoader.loadClass(javax.mail.MessagingException): calling
componentClassLoader.loadClass()
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] ERROR
org.apache.ranger.admin.client.RangerAdminRESTClient - Failed to get response,
Error is : null
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.admin.client.RangerAdminRESTClient - <==
RangerAdminRESTClient.getRangerRolesDownloadResponse(-1, 1728128759797): null
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] ERROR
org.apache.ranger.admin.client.RangerAdminRESTClient - Error getting Roles;
Received NULL response!!. secureMode=true, user=hive/master-03....@realm.tdp
(auth:KERBEROS), serviceName=hive-tdp
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.admin.client.RangerAdminRESTClient - <==
RangerAdminRESTClient.getRolesIfUpdatedWithCred(-1, 1728128759797): null
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.admin.client.RangerAdminRESTClient - <==
RangerAdminRESTClient.getRolesIfUpdated(-1, 1728128759797):
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.plugin.util.RangerRolesProvider -
RangerRolesProvider(serviceName=hive-tdp).run(): no update found.
lastKnownRoleVersion=-1
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.perf.policyengine.init -
[PERF]:PolicyRefresher(serviceName=hive-tdp)-36:RangerRolesProvider.loadUserGroupRolesFromAdmin(serviceName=hive-tdp):177226260:214664927
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.plugin.util.RangerRolesProvider - <==
RangerRolesProvider(serviceName=hive-tdp serviceType= hive
).loadUserGroupRolesFromAdmin()
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.perf.policyengine.init - In-Use memory: 307461672, Free
memory:71074264
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.perf.policyengine.init -
[PERF]:PolicyRefresher(serviceName=hive-tdp)-36:RangerRolesProvider.loadUserGroupRoles(serviceName=hive-tdp):177292518:214730011
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.plugin.util.RangerRolesProvider - <==
RangerRolesProvider(serviceName=hive-tdp).loadUserGroupRoles()
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.plugin.util.PolicyRefresher - <==
PolicyRefresher(serviceName=hive-tdp).loadRoles()
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.plugin.util.PolicyRefresher - ==>
PolicyRefresher(serviceName=hive-tdp).loadPolicy()
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.perf.policyengine.init - In-Use memory: 307461672, Free
memory:71074264
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.plugin.util.PolicyRefresher - ==>
PolicyRefresher(serviceName=hive-tdp).loadPolicyfromPolicyAdmin()
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.admin.client.RangerAdminRESTClient - ==>
RangerAdminRESTClient.getServicePoliciesIfUpdated(-1, 1728128790697)
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.admin.client.RangerAdminRESTClient - ==>
RangerAdminRESTClient.getServicePoliciesIfUpdatedWithCred(-1, 1728128790697)
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.admin.client.RangerAdminRESTClient - ==>
RangerAdminRESTClient.getRangerAdminPolicyDownloadResponse(-1, 1728128790697)
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.ranger.admin.client.RangerAdminRESTClient - Checking Service policy
if updated as user : hive/master-03....@realm.tdp (auth:KERBEROS)
12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG
org.apache.hadoop.security.UserGroupInformation - PrivilegedAction [as:
hive/master-03....@realm.tdp (auth:KERBEROS)][action:
org.apache.ranger.admin.client.RangerAdminRESTClient$13@ab6302e]
java.lang.Exception: null
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1873)
at
org.apache.ranger.admin.client.RangerAdminRESTClient.getRangerAdminPolicyDownloadResponse(RangerAdminRESTClient.java:948)
at
org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdatedWithCred(RangerAdminRESTClient.java:823)
at
org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:127)
at
org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:302)
at
org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:241)
at
org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:203)
{code}
Test connection on the ranger admin UI : !image-2024-10-05-14-21-23-474.png!
My ranger logs :
{code:java}
2024-10-05 12:06:45,082 [https-jsse-nio-6182-exec-13] INFO
[RangerTransactionService.java:156] RangerTransactionService:
tasks(scheduled=52, executed=51, failed=0, pending=1)
2024-10-05 12:06:47,733 [https-jsse-nio-6182-exec-9] INFO
[TagDBStore.java:1379] SUPPORTS_TAG_DELTAS=false
2024-10-05 12:06:47,734 [https-jsse-nio-6182-exec-9] INFO
[TagDBStore.java:1380] SUPPORTS_IN_PLACE_TAG_UPDATES=false
2024-10-05 12:20:44,433 [https-jsse-nio-6182-exec-6] INFO
[SpringEventListener.java:76] Login Successful:admin | Ip Address:192.168.56.1
| sessionId=ED3F33CD1C50E5B70088756FA2D196C3 | Epoch=1728130844433
2024-10-05 12:20:52,264 [timed-executor-pool-0] INFO [BaseClient.java:132] Init
Lookup Login: security enabled, using lookupPrincipal/lookupKeytab
2024-10-05 12:20:52,271 [timed-executor-pool-0] INFO [HiveClient.java:85]
Secured Mode: JDBC Connection done with preAuthenticated Subject
2024-10-05 12:20:52,274 [timed-executor-pool-0-SendThread(master-03.tdp:2181)]
WARN [ClientCnxn.java:1163] SASL configuration failed. Will continue connection
to Zookeeper server without SASL authentication, if Zookeeper server allows it.
javax.security.auth.login.LoginException: No JAAS configuration section named
'Client' was found in specified JAAS configuration file: '/dev/null'.
at
org.apache.zookeeper.client.ZooKeeperSaslClient.<init>(ZooKeeperSaslClient.java:190)
at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1157)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1207)
2024-10-05 12:20:52,275 [timed-executor-pool-0-EventThread] ERROR
[ConnectionState.java:247] Authentication failed
2024-10-05 12:20:52,571 [timed-executor-pool-0-SendThread(master-03.tdp:2181)]
WARN [ClientCnxn.java:1163] SASL configuration failed. Will continue connection
to Zookeeper server without SASL authentication, if Zookeeper server allows it.
javax.security.auth.login.LoginException: No JAAS configuration section named
'Client' was found in specified JAAS configuration file: '/dev/null'.
at
org.apache.zookeeper.client.ZooKeeperSaslClient.<init>(ZooKeeperSaslClient.java:190)
at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1157)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1207)
2024-10-05 12:20:52,572 [timed-executor-pool-0-EventThread] ERROR
[ConnectionState.java:247] Authentication failed
2024-10-05 12:20:53,387 [timed-executor-pool-0] WARN [HiveConnection.java:237]
Failed to connect to master-03:10001
2024-10-05 12:20:53,391 [timed-executor-pool-0-SendThread(master-01.tdp:2181)]
WARN [ClientCnxn.java:1163] SASL configuration failed. Will continue connection
to Zookeeper server without SASL authentication, if Zookeeper server allows it.
javax.security.auth.login.LoginException: No JAAS configuration section named
'Client' was found in specified JAAS configuration file: '/dev/null'.
at
org.apache.zookeeper.client.ZooKeeperSaslClient.<init>(ZooKeeperSaslClient.java:190)
at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1157)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1207)
2024-10-05 12:20:53,392 [timed-executor-pool-0-EventThread] ERROR
[ConnectionState.java:247] Authentication failed
2024-10-05 12:20:53,523 [timed-executor-pool-0] WARN [HiveConnection.java:264]
Could not open client transport with JDBC Uri:
jdbc:hive2://master-03:10001/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;sslTrustStore=/etc/ssl/certs/truststore.jks;trustStorePassword=Truststore123!:
Failed to open new session: org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:No privilege 'Select' found for inputs {
database:default}) Retrying 0 of 1
2024-10-05 12:20:54,903 [timed-executor-pool-0] WARN [HiveConnection.java:237]
Failed to connect to master-02:10001
2024-10-05 12:20:54,908 [timed-executor-pool-0-SendThread(master-02.tdp:2181)]
WARN [ClientCnxn.java:1163] SASL configuration failed. Will continue connection
to Zookeeper server without SASL authentication, if Zookeeper server allows it.
javax.security.auth.login.LoginException: No JAAS configuration section named
'Client' was found in specified JAAS configuration file: '/dev/null'.
at
org.apache.zookeeper.client.ZooKeeperSaslClient.<init>(ZooKeeperSaslClient.java:190)
at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1157)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1207)
2024-10-05 12:20:54,908 [timed-executor-pool-0-EventThread] ERROR
[ConnectionState.java:247] Authentication failed
2024-10-05 12:20:55,035 [timed-executor-pool-0] ERROR [Utils.java:619] Unable
to read HiveServer2 configs from ZooKeeper
2024-10-05 12:20:55,040 [timed-executor-pool-0-SendThread(master-03.tdp:2181)]
WARN [ClientCnxn.java:1163] SASL configuration failed. Will continue connection
to Zookeeper server without SASL authentication, if Zookeeper server allows it.
javax.security.auth.login.LoginException: No JAAS configuration section named
'Client' was found in specified JAAS configuration file: '/dev/null'.
at
org.apache.zookeeper.client.ZooKeeperSaslClient.<init>(ZooKeeperSaslClient.java:190)
at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1157)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1207)
2024-10-05 12:20:55,041 [timed-executor-pool-0-EventThread] ERROR
[ConnectionState.java:247] Authentication failed
2024-10-05 12:20:55,321 [timed-executor-pool-0] ERROR [HiveClient.java:570]
Unable to Connect to Hive
org.apache.ranger.plugin.client.HadoopException: Unable to connect to Hive
Thrift Server instance.
at
org.apache.ranger.services.hive.client.HiveClient.initConnection(HiveClient.java:729)
at
org.apache.ranger.services.hive.client.HiveClient.initConnection(HiveClient.java:568)
at
org.apache.ranger.services.hive.client.HiveClient.access$000(HiveClient.java:56)
at
org.apache.ranger.services.hive.client.HiveClient$1.run(HiveClient.java:88)
at
org.apache.ranger.services.hive.client.HiveClient$1.run(HiveClient.java:86)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.ranger.services.hive.client.HiveClient.initHive(HiveClient.java:86)
at
org.apache.ranger.services.hive.client.HiveClient.<init>(HiveClient.java:77)
at
org.apache.ranger.services.hive.client.HiveClient.connectionTest(HiveClient.java:827)
at
org.apache.ranger.services.hive.client.HiveResourceMgr.connectionTest(HiveResourceMgr.java:49)
at
org.apache.ranger.services.hive.RangerServiceHive.validateConfig(RangerServiceHive.java:82)
at
org.apache.ranger.biz.ServiceMgr$ValidateCallable.actualCall(ServiceMgr.java:732)
at
org.apache.ranger.biz.ServiceMgr$ValidateCallable.actualCall(ServiceMgr.java:719)
at org.apache.ranger.biz.ServiceMgr$TimedCallable.call(ServiceMgr.java:680)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.sql.SQLException:
org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2
configs from ZooKeeper
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:170)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:270)
at
org.apache.ranger.services.hive.client.HiveClient.initConnection(HiveClient.java:712)
... 18 common frames omitted
Caused by: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read
HiveServer2 configs from ZooKeeper
at
org.apache.hive.jdbc.ZooKeeperHiveClientHelper.configureConnParams(ZooKeeperHiveClientHelper.java:147)
at
org.apache.hive.jdbc.Utils.configureConnParamsFromZooKeeper(Utils.java:511)
at org.apache.hive.jdbc.Utils.parseURL(Utils.java:334)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:168)
... 22 common frames omitted
Caused by: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read
HiveServer2 uri from ZooKeeper:
at
org.apache.hive.jdbc.ZooKeeperHiveClientHelper.updateParamsWithZKServerNode(ZooKeeperHiveClientHelper.java:125)
at
org.apache.hive.jdbc.ZooKeeperHiveClientHelper.configureConnParams(ZooKeeperHiveClientHelper.java:145)
... 25 common frames omitted
2024-10-05 12:20:55,321 [timed-executor-pool-0] ERROR [HiveResourceMgr.java:51]
<== HiveResourceMgr.connectionTest Error:
org.apache.ranger.plugin.client.HadoopException: Unable to connect to Hive
Thrift Server instance.
2024-10-05 12:20:55,322 [timed-executor-pool-0] ERROR
[RangerServiceHive.java:84] <== RangerServiceHive.validateConfig
Error:org.apache.ranger.plugin.client.HadoopException: Unable to connect to
Hive Thrift Server instance.
2024-10-05 12:20:55,322 [timed-executor-pool-0] ERROR [ServiceMgr.java:682]
TimedCallable.call: Error:org.apache.ranger.plugin.client.HadoopException:
Unable to connect to Hive Thrift Server instance.
2024-10-05 12:20:55,322 [https-jsse-nio-6182-exec-4] ERROR
[ServiceMgr.java:231] ==> ServiceMgr.validateConfig
Error:org.apache.ranger.plugin.client.HadoopException:
org.apache.ranger.plugin.client.HadoopException: Unable to connect to Hive
Thrift Server instance.
{code}
It tells me that i have No JAAS configuration section named 'Client' but I have
one configured in my zookeeper edge node.
I don't now what's the issue, if anyone can help me.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
