[
https://issues.apache.org/jira/browse/AMBARI-12096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zack Marsh resolved AMBARI-12096.
---------------------------------
Resolution: Not A Problem
This issue was being caused by incorrect proxy user configuration. Closing.
> Hive Service Check failing after enabling Kerberos
> --------------------------------------------------
>
> Key: AMBARI-12096
> URL: https://issues.apache.org/jira/browse/AMBARI-12096
> Project: Ambari
> Issue Type: Bug
> Environment: ambari-2.1.0-1213, hdp-2.3.0.0-2450, sles11sp3
> Reporter: Zack Marsh
> Priority: Critical
>
> After enabling Kerberos, the Hive Service Check is consistently failing.
> Errors as seen in Ambari when the Service Check fails:
> stderr:
> {code}
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py",
> line 106, in <module>
> HiveServiceCheck().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 216, in execute
> method(env)
> File
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py",
> line 102, in service_check
> hcat_service_check()
> File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py",
> line 89, in thunk
> return fn(*args, **kwargs)
> File
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hcat_service_check.py",
> line 65, in hcat_service_check
> logoutput=True)
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py",
> line 157, in __init__
> self.env.run()
> File
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
> line 152, in run
> self.run_action(resource, action)
> File
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
> line 118, in run_action
> provider_action()
> File
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
> line 254, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 70, in inner
> result = function(command, **kwargs)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 92, in checked_call
> tries=tries, try_sleep=try_sleep)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 291, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt
> /etc/security/keytabs/smokeuser.headless.keytab ambari-qa; env
> JAVA_HOME=/opt/teradata/jvm64/jdk8
> /var/lib/ambari-agent/data/tmp/hcatSmoke.sh hcatsmokeid00270328_date262315
> prepare' returned 64. kinit(v5): Client not found in Kerberos database while
> getting initial credentials
> WARNING: Use "yarn jar" to launch YARN applications.
> FAILED: SemanticException
> MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
> Unauthorized connection for super-user:
> hive/piripiri2.labs.teradata....@piripiri.hadoop.teradata.com from IP
> 39.0.40.2)
> WARNING: Use "yarn jar" to launch YARN applications.
> OK
> Time taken: 1.612 seconds
> WARNING: Use "yarn jar" to launch YARN applications.
> FAILED: SemanticException
> MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
> Unauthorized connection for super-user:
> hive/piripiri2.labs.teradata....@piripiri.hadoop.teradata.com from IP
> 39.0.40.2)
> {code}
> stdout:
> {code}
> Test connectivity to hive server
> Waiting for the Hive server to start...
> 2015-06-23 12:26:21,859 - Execute['/usr/bin/kinit -kt
> /etc/security/keytabs/smokeuser.headless.keytab
> ambari-qa-pirip...@piripiri.hadoop.teradata.com; '] {'user': 'ambari-qa'}
> 2015-06-23 12:26:21,910 - Execute['! beeline -u
> 'jdbc:hive2://piripiri2.labs.teradata.com:10000/;transportMode=binary;principal=hive/_h...@piripiri.hadoop.teradata.com'
> -e '' 2>&1| awk '{print}'|grep -i -e 'Connection refused' -e 'Invalid URL'']
> {'path': ['/bin/', '/usr/bin/', '/usr/lib/hive/bin/', '/usr/sbin/'], 'user':
> 'ambari-qa', 'timeout': 30}
> Successfully connected to piripiri2.labs.teradata.com on port 10000
> Successfully connected to Hive at piripiri3.labs.teradata.com on port 10000
> after 3 seconds
> 2015-06-23 12:26:25,609 - File['/var/lib/ambari-agent/data/tmp/hcatSmoke.sh']
> {'content': StaticFile('hcatSmoke.sh'), 'mode': 0755}
> 2015-06-23 12:26:25,610 - Writing
> File['/var/lib/ambari-agent/data/tmp/hcatSmoke.sh'] because it doesn't exist
> 2015-06-23 12:26:25,610 - Changing permission for
> /var/lib/ambari-agent/data/tmp/hcatSmoke.sh from 644 to 755
> 2015-06-23 12:26:25,610 - Execute['/usr/bin/kinit -kt
> /etc/security/keytabs/smokeuser.headless.keytab ambari-qa; env
> JAVA_HOME=/opt/teradata/jvm64/jdk8
> /var/lib/ambari-agent/data/tmp/hcatSmoke.sh hcatsmokeid00270328_date262315
> prepare'] {'logoutput': True, 'path': ['/usr/sbin', '/usr/local/bin', '/bin',
> '/usr/bin',
> '/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/sbin:/sbin:/usr/lib/ambari-server/*:/sbin:/usr/sbin:/usr/local/sbin:/root/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/teradata/bynet/bin:/opt/teradata/jvm64/jdk8/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:.:/opt/teradata/sm3g/bin:/opt/dell/srvadmin/bin:/opt/dell/srvadmin/sbin:/opt/teradata/dswap/sbin:/usr/tdbms/bin:/opt/teradata/gsctools/bin:/opt/teradata/vmf/bin:/opt/teradata/bin:/var/lib/ambari-agent:/var/lib/ambari-agent:/usr/hdp/current/hive-client/bin:/usr/hdp/current/hadoop-client/bin'],
> 'tries': 3, 'user': 'ambari-qa', 'try_sleep': 5}
> kinit(v5): Client not found in Kerberos database while getting initial
> credentials
> WARNING: Use "yarn jar" to launch YARN applications.
> FAILED: SemanticException
> MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
> Unauthorized connection for super-user:
> hive/piripiri2.labs.teradata....@piripiri.hadoop.teradata.com from IP
> 39.0.40.2)
> WARNING: Use "yarn jar" to launch YARN applications.
> OK
> Time taken: 1.448 seconds
> WARNING: Use "yarn jar" to launch YARN applications.
> FAILED: SemanticException
> MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
> Unauthorized connection for super-user:
> hive/piripiri2.labs.teradata....@piripiri.hadoop.teradata.com from IP
> 39.0.40.2)
> 2015-06-23 12:26:39,610 - Retrying after 5 seconds. Reason: Execution of
> '/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab
> ambari-qa; env JAVA_HOME=/opt/teradata/jvm64/jdk8
> /var/lib/ambari-agent/data/tmp/hcatSmoke.sh hcatsmokeid00270328_date262315
> prepare' returned 64. kinit(v5): Client not found in Kerberos database while
> getting initial credentials
> WARNING: Use "yarn jar" to launch YARN applications.
> FAILED: SemanticException
> MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
> Unauthorized connection for super-user:
> hive/piripiri2.labs.teradata....@piripiri.hadoop.teradata.com from IP
> 39.0.40.2)
> WARNING: Use "yarn jar" to launch YARN applications.
> OK
> Time taken: 1.448 seconds
> WARNING: Use "yarn jar" to launch YARN applications.
> FAILED: SemanticException
> MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
> Unauthorized connection for super-user:
> hive/piripiri2.labs.teradata....@piripiri.hadoop.teradata.com from IP
> 39.0.40.2)
> kinit(v5): Client not found in Kerberos database while getting initial
> credentials
> WARNING: Use "yarn jar" to launch YARN applications.
> FAILED: SemanticException
> MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
> Unauthorized connection for super-user:
> hive/piripiri2.labs.teradata....@piripiri.hadoop.teradata.com from IP
> 39.0.40.2)
> WARNING: Use "yarn jar" to launch YARN applications.
> OK
> Time taken: 1.42 seconds
> WARNING: Use "yarn jar" to launch YARN applications.
> FAILED: SemanticException
> MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
> Unauthorized connection for super-user:
> hive/piripiri2.labs.teradata....@piripiri.hadoop.teradata.com from IP
> 39.0.40.2)
> 2015-06-23 12:26:58,283 - Retrying after 5 seconds. Reason: Execution of
> '/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab
> ambari-qa; env JAVA_HOME=/opt/teradata/jvm64/jdk8
> /var/lib/ambari-agent/data/tmp/hcatSmoke.sh hcatsmokeid00270328_date262315
> prepare' returned 64. kinit(v5): Client not found in Kerberos database while
> getting initial credentials
> WARNING: Use "yarn jar" to launch YARN applications.
> FAILED: SemanticException
> MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
> Unauthorized connection for super-user:
> hive/piripiri2.labs.teradata....@piripiri.hadoop.teradata.com from IP
> 39.0.40.2)
> WARNING: Use "yarn jar" to launch YARN applications.
> OK
> Time taken: 1.42 seconds
> WARNING: Use "yarn jar" to launch YARN applications.
> FAILED: SemanticException
> MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
> Unauthorized connection for super-user:
> hive/piripiri2.labs.teradata....@piripiri.hadoop.teradata.com from IP
> 39.0.40.2)
> kinit(v5): Client not found in Kerberos database while getting initial
> credentials
> WARNING: Use "yarn jar" to launch YARN applications.
> FAILED: SemanticException
> MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
> Unauthorized connection for super-user:
> hive/piripiri2.labs.teradata....@piripiri.hadoop.teradata.com from IP
> 39.0.40.2)
> WARNING: Use "yarn jar" to launch YARN applications.
> OK
> Time taken: 1.612 seconds
> WARNING: Use "yarn jar" to launch YARN applications.
> FAILED: SemanticException
> MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
> Unauthorized connection for super-user:
> hive/piripiri2.labs.teradata....@piripiri.hadoop.teradata.com from IP
> 39.0.40.2)
> {code}
> The cluster has Hive HA enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
