I don't have any particular reason for selecting arcfour encryption type. If I
need to change it and it will work I can do.
Values from krb5.conf:
[Libdefaults]
default_realm = ADS.AUTODESK.COM
krb4_config = /etc/krb.conf
krb4_realms = /etc/krb.realms
kdc_timesync = 1
ccache_type = 4
forwardable = true
proxiable = true
v4_instance_resolve = false
v4_name_convert = {
host = {
rcmd = host
ftp = ftp
}
plain = {
something = something-else
}
}
fcc-mit-ticketflags = true
default_tkt_enctypes = RC4 HMAC-des-cbc-crc of-CBC-MD5 AES256-CTS
default_tgs_enctypes = RC4-HMAC des-cbc-crc des-cbc-md5 AES256-CTS
[realms]
ADS.AUTODESK.COM = {
kdc = krb.ads.autodesk.com: 88
admin_server = krb.ads.autodesk.com
default_domain = ads.autodesk.com
database_module = openldap_ldapconf
master_key_type = aes256-cts
supported_enctypes = aes256-cts:normal aes128-cts:normal
des3-hmac-sha1:normal arcfour-hmac:normal des-hmac-sha1:normal
des-cbc-md5:normal des-cbc-crc:normal
default_principal_flags = +preauth
}
Thanks so much for your help,
Ricardo.
________________________________
From: Vivek Shrivastava <[email protected]>
Sent: Monday, January 30, 2017 11:01:24 AM
To: [email protected]
Subject: Re: Pls Help me - Hive Kerberos Issue
Any particular reason for selecting arcfour encryption type? Could you please
post defaults (e.g enc_type) values from krb5.conf
On Mon, Jan 30, 2017 at 10:57 AM, Ricardo Fajardo
<[email protected]<mailto:[email protected]>> wrote:
1. klist -fe
[cloudera@quickstart bin]$ klist -fe
Ticket cache: FILE:/tmp/krb5cc_501
Default principal: [email protected]<mailto:[email protected]>
Valid starting Expires Service principal
01/30/17 10:52:37 01/30/17 20:52:43
krbtgt/[email protected]<mailto:[email protected]>
renew until 01/31/17 10:52:37, Flags: FPRIA
Etype (skey, tkt): arcfour-hmac, arcfour-hmac
[cloudera@quickstart bin]$
2. relevant entries from HiveServer2 log
beeline> !connect
jdbc:hive2://localhost:10000/default;principal=hive/[email protected]<mailto:[email protected]>;hive.server2.proxy.user=t_fajar
!connect jdbc:hive2://localhost:10000/default;principal=hive/_HOST@ADS.
AUTODESK.COM<http://AUTODESK.COM>;hive.server2.proxy.user=t_fajar
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/home/cloudera/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.6.2/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/home/cloudera/.m2/repository/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/home/cloudera/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to
jdbc:hive2://localhost:10000/default;principal=hive/[email protected]<mailto:[email protected]>;hive.server2.proxy.user=t_fajar
17/01/27 16:16:36 INFO Utils: Supplied authorities: localhost:10000
17/01/27 16:16:36 INFO Utils: Resolved authority: localhost:10000
17/01/27 16:16:36 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with
annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time,
value=[Rate of successful kerberos logins and latency (milliseconds)], about=,
type=DEFAULT, always=false, sampleName=Ops)
17/01/27 16:16:36 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with
annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time,
value=[Rate of failed kerberos logins and latency (milliseconds)], about=,
type=DEFAULT, always=false, sampleName=Ops)
17/01/27 16:16:36 DEBUG MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time,
value=[GetGroups], about=, type=DEFAULT, always=false, sampleName=Ops)
17/01/27 16:16:36 DEBUG MetricsSystemImpl: UgiMetrics, User and group related
metrics
17/01/27 16:16:37 DEBUG Shell: setsid exited with exit code 0
17/01/27 16:16:37 DEBUG Groups: Creating new Groups object
17/01/27 16:16:37 DEBUG NativeCodeLoader: Trying to load the custom-built
native-hadoop library...
17/01/27 16:16:37 DEBUG NativeCodeLoader: Failed to load native-hadoop with
error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
17/01/27 16:16:37 DEBUG NativeCodeLoader:
java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
17/01/27 16:16:37 WARN NativeCodeLoader: Unable to load native-hadoop library
for your platform... using builtin-java classes where applicable
17/01/27 16:16:37 DEBUG PerformanceAdvisory: Falling back to shell based
17/01/27 16:16:37 DEBUG JniBasedUnixGroupsMappingWithFallback: Group mapping
impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
17/01/27 16:16:38 DEBUG Groups: Group mapping
impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
cacheTimeout=300000; warningDeltaMs=5000
17/01/27 16:16:38 DEBUG UserGroupInformation: hadoop login
17/01/27 16:16:38 DEBUG UserGroupInformation: hadoop login commit
17/01/27 16:16:38 DEBUG UserGroupInformation: using local user:UnixPrincipal:
cloudera
17/01/27 16:16:38 DEBUG UserGroupInformation: Using user: "UnixPrincipal:
cloudera" with name cloudera
17/01/27 16:16:38 DEBUG UserGroupInformation: User entry: "cloudera"
17/01/27 16:16:56 DEBUG UserGroupInformation: UGI loginUser:cloudera
(auth:SIMPLE)
17/01/27 16:16:56 DEBUG HadoopThriftAuthBridge: Current authMethod = SIMPLE
17/01/27 16:16:56 DEBUG HadoopThriftAuthBridge: Setting UGI conf as passed-in
authMethod of kerberos != current.
17/01/30 10:24:45 DEBUG UserGroupInformation: PrivilegedAction as:cloudera
(auth:SIMPLE)
from:org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Client.createClientTransport(HadoopThriftAuthBridge.java:208)
17/01/30 10:55:02 DEBUG UserGroupInformation: PrivilegedAction as:cloudera
(auth:SIMPLE)
from:org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
17/01/30 10:55:02 DEBUG TSaslTransport: opening transport
org.apache.thrift.transport.TSaslClientTransport@1119f7c5
17/01/30 10:55:02 ERROR TSaslTransport: SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed
at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
~[?:1.7.0_67]
at
org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
~[libthrift-0.9.3.jar:0.9.3]
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
[libthrift-0.9.3.jar:0.9.3]
at
org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
[libthrift-0.9.3.jar:0.9.3]
at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
[classes/:?]
at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:1)
[classes/:?]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.7.0_67]
at javax.security.auth.Subject.doAs(Subject.java:415) [?:1.7.0_67]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
[hadoop-common-2.7.2.jar:?]
at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
[classes/:?]
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:227)
[classes/:?]
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:182)
[classes/:?]
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107) [classes/:?]
at java.sql.DriverManager.getConnection(DriverManager.java:571) [?:1.7.0_67]
at java.sql.DriverManager.getConnection(DriverManager.java:187) [?:1.7.0_67]
at
org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:145)
[classes/:?]
at
org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:209)
[classes/:?]
at org.apache.hive.beeline.Commands.connect(Commands.java:1524) [classes/:?]
at org.apache.hive.beeline.Commands.connect(Commands.java:1419) [classes/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_67]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
~[?:1.7.0_67]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.7.0_67]
at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_67]
at
org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:56)
[classes/:?]
at org.apache.hive.beeline.BeeLine.execCommandWithPrefix(BeeLine.java:1127)
[classes/:?]
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1166) [classes/:?]
at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:999) [classes/:?]
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:909) [classes/:?]
at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:511)
[classes/:?]
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:494) [classes/:?]
Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism
level: Failed to find any Kerberos tgt)
at
sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
~[?:1.7.0_67]
at
sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
~[?:1.7.0_67]
at
sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
~[?:1.7.0_67]
at
sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
~[?:1.7.0_67]
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
~[?:1.7.0_67]
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
~[?:1.7.0_67]
at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
~[?:1.7.0_67]
... 29 more
17/01/30 10:55:02 DEBUG TSaslTransport: CLIENT: Writing message with status BAD
and payload length 19
17/01/30 10:55:02 WARN HiveConnection: Failed to connect to localhost:10000
HS2 may be unavailable, check server status
Error: Could not open client transport with JDBC Uri:
jdbc:hive2://localhost:10000/default;principal=hive/[email protected]<mailto:[email protected]>;hive.server2.proxy.user=t_fajar:
GSS initiate failed (state=08S01,code=0)
beeline>
________________________________
From: Vivek Shrivastava
<[email protected]<mailto:[email protected]>>
Sent: Monday, January 30, 2017 10:48:35 AM
To: [email protected]<mailto:[email protected]>
Subject: Re: Pls Help me - Hive Kerberos Issue
Please paste the output of
1. klist -fe
2. relevant entries from HiveServer2 log
On Mon, Jan 30, 2017 at 10:11 AM, Ricardo Fajardo
<[email protected]<mailto:[email protected]>> wrote:
I could not resolve the problem.
I have debugged the code and I found out that:
1. On the org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge class line 208
....
UserGroupInformation.getCurrentUser return (). Two (....
..
This method always returns the user of the operative system but and I need
authenticate the user set on the property: hive.server2.proxy.user=yourid
because I have a token for this one.
2. I have found out that the hive.server2.proxy.user is implemented on the
org.apache.hive.jdbc.HiveConnection class method: openSession() but this code
is never executed.
3. On the org.apache.hive.service.auth.HiveAuthFactory class there is this code
on the method getAuthTransFactory():
....
if (authTypeStr.equalsIgnoreCase(AuthTypes.KERBEROS.getAuthName())) {
// no-op
....
It means that Kerberos authentication is not implemented?
Please anyone can help me??
Thanks,
Ricardo.
________________________________
From: Dulam, Naresh
<[email protected]<mailto:[email protected]>>
Sent: Thursday, January 26, 2017 8:41:48 AM
To: [email protected]<mailto:[email protected]>
Subject: RE: Pls Help me - Hive Kerberos Issue
Kinit yourid -k -t your.keytab [email protected]<mailto:[email protected]>
# Connect using following JDBC connection string
#
jdbc:hive2://myHost.myOrg.com:10000/default;principal=hive/[email protected];hive.server2.proxy.user=yourid<http://myHost.myOrg.com:10000/default;principal=hive/[email protected];hive.server2.proxy.user=yourid>
From: Ricardo Fajardo
[mailto:[email protected]<mailto:[email protected]>]
Sent: Thursday, January 26, 2017 1:37 AM
To: [email protected]<mailto:[email protected]>
Subject: Pls Help me - Hive Kerberos Issue
Hello,
Please I need your help with the Kerberos authentication with Hive.
I am following this guide:
https://www.cloudera.com/documentation/enterprise/5-4-x/topics/cdh_sg_hiveserver2_security.html#topic_9_1_1
But I am getting this error:
Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism
level: Failed to find any Kerberos tgt)
I have a remote Kerberos server and I can generate a token with kinit for my
user. I created a keytab file with my passwd for my user. Please tell me if it
is ok.
On the another hand when I am debugging the hive code the operative system user
is authenticated but I need authenticate my Kerberos user, can you tell me how
I can achieve that? How can I store my tickets where Hive can load it?? or How
can I verify where Hive is searching the tickets and what Hive is reading??
Thanks so much for your help.
Best regards,
Ricardo.
________________________________
This message, and any attachments, is for the intended recipient(s) only, may
contain information that is privileged, confidential and/or proprietary and
subject to important terms and conditions available at
http://www.bankofamerica.com/emaildisclaimer. If you are not the intended
recipient, please delete this message.
