[ https://issues.apache.org/jira/browse/HIVE-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295470#comment-15295470 ]
Vijay Singh commented on HIVE-13817: ------------------------------------ Please see the following SPN values available to HiveServer2. {code} [root@vjd-1 ~]# klist -kt /var/run/cloudera-scm-agent/process/2121-hive-HIVESERVER2/hive.keytab Keytab name: FILE:/var/run/cloudera-scm-agent/process/2121-hive-HIVESERVER2/hive.keytab KVNO Timestamp Principal ---- ------------------- ------------------------------------------------------ 1 05/22/2016 01:07:19 hive/vjtest-local.gce.cloudera....@gce.cloudera.com 1 05/22/2016 01:07:19 hive/vjtest-local.gce.cloudera....@gce.cloudera.com 1 05/22/2016 01:07:19 hive/vjtest-local.gce.cloudera....@gce.cloudera.com 1 05/22/2016 01:07:19 hive/vjd-1.gce.cloudera....@gce.cloudera.com 1 05/22/2016 01:07:19 hive/vjd-1.gce.cloudera....@gce.cloudera.com 1 05/22/2016 01:07:19 hive/vjd-1.gce.cloudera....@gce.cloudera.com 1 05/22/2016 01:07:19 HTTP/vjd-1.gce.cloudera....@gce.cloudera.com 1 05/22/2016 01:07:19 HTTP/vjd-1.gce.cloudera....@gce.cloudera.com 1 05/22/2016 01:07:19 HTTP/vjd-1.gce.cloudera....@gce.cloudera.com {code} Please see the following program that determines canonical name. {code} [root@vjd-1 ~]# java NSLookupFwd vjlb Host: vjlb Canonical HostName: vjtest-local.gce.cloudera.com {code} Please find the log excerpt from executing beeline with new driver with cname alias functionality turned on in connection string. {code} [root@vjd-1 ~]# hadoop version Hadoop 2.6.0-cdh5.7.0 Subversion http://github.com/cloudera/hadoop -r c00978c67b0d3fe9f3b896b5030741bd40bf541a Compiled by jenkins on 2016-03-23T18:36Z Compiled with protoc 2.5.0 >From source with checksum b2eabfa328e763c88cb14168f9b372 This command was run using /opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/jars/hadoop-common-2.6.0-cdh5.7.0.jar [root@vjd-1 ~]# beeline -u "jdbc:hive2://vjlb:10000/default;principal=hive/_h...@gce.cloudera.com;useCanonicalName=true" -e "show tables;" Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0 Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release 16/05/22 01:14:45 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree module jar containing PrefixTreeCodec is not present. Continuing without it. Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0 scan complete in 2ms Connecting to jdbc:hive2://vjlb:10000/default;principal=hive/_h...@gce.cloudera.com;useCanonicalName=true Connected to: Apache Hive (version 1.1.0-cdh5.7.0) Driver: Hive JDBC (version 1.1.0-cdh5.7.0) Transaction isolation: TRANSACTION_REPEATABLE_READ INFO : Compiling command(queryId=hive_20160522011414_690deeda-df58-469d-b18f-33890162cf6f): show tables INFO : Semantic Analysis Completed INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null) INFO : Completed compiling command(queryId=hive_20160522011414_690deeda-df58-469d-b18f-33890162cf6f); Time taken: 0.069 seconds INFO : Executing command(queryId=hive_20160522011414_690deeda-df58-469d-b18f-33890162cf6f): show tables INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20160522011414_690deeda-df58-469d-b18f-33890162cf6f); Time taken: 0.163 seconds INFO : OK +------------+--+ | tab_name | +------------+--+ | customers | | sample_07 | | sample_08 | | web_logs | +------------+--+ 4 rows selected (0.376 seconds) Beeline version 1.1.0-cdh5.7.0 by Apache Hive Closing: 0: jdbc:hive2://vjlb:10000/default;principal=hive/_h...@gce.cloudera.com;useCanonicalName=true [root@vjd-1 ~]# {code} Please find the log excerpt for connection without canonical name functionality turned on in connection string. {code} [root@vjd-1 ~]# beeline -u "jdbc:hive2://vjlb:10000/default;principal=hive/_h...@gce.cloudera.com" -e "show tables;" Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0 Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release 16/05/22 01:20:41 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree module jar containing PrefixTreeCodec is not present. Continuing without it. Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0 scan complete in 3ms Connecting to jdbc:hive2://vjlb:10000/default;principal=hive/_h...@gce.cloudera.com 16/05/22 01:20:43 [main]: ERROR transport.TSaslTransport: SASL negotiation failure javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7))] at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:209) at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:181) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:208) at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:137) at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:178) at org.apache.hive.beeline.Commands.connect(Commands.java:1335) at org.apache.hive.beeline.Commands.connect(Commands.java:1256) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:52) at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1081) at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:760) at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:823) at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:482) at org.apache.hive.beeline.BeeLine.main(BeeLine.java:465) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7)) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:770) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) ... 34 more Caused by: KrbException: Server not found in Kerberos database (7) at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:70) at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:259) at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:270) at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:302) at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:120) at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:458) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:693) ... 37 more Caused by: KrbException: Identifier doesn't match expected value (906) at sun.security.krb5.internal.KDCRep.init(KDCRep.java:140) at sun.security.krb5.internal.TGSRep.init(TGSRep.java:65) at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:60) at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55) ... 43 more Error: Could not open client transport with JDBC Uri: jdbc:hive2://vjlb:10000/default;principal=hive/_h...@gce.cloudera.com: GSS initiate failed (state=08S01,code=0) 16/05/22 01:20:44 [main]: ERROR transport.TSaslTransport: SASL negotiation failure javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7))] at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:209) at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:181) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:208) at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:137) at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:178) at org.apache.hive.beeline.BeeLine.assertConnection(BeeLine.java:1306) at org.apache.hive.beeline.Commands.execute(Commands.java:1041) at org.apache.hive.beeline.Commands.sql(Commands.java:976) at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1085) at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:773) at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:823) at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:482) at org.apache.hive.beeline.BeeLine.main(BeeLine.java:465) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7)) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:770) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) ... 30 more Caused by: KrbException: Server not found in Kerberos database (7) at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:70) at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:259) at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:270) at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:302) at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:120) at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:458) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:693) ... 33 more Caused by: KrbException: Identifier doesn't match expected value (906) at sun.security.krb5.internal.KDCRep.init(KDCRep.java:140) at sun.security.krb5.internal.TGSRep.init(TGSRep.java:65) at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:60) at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55) ... 39 more No current connection 16/05/22 01:20:44 [main]: ERROR transport.TSaslTransport: SASL negotiation failure javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7))] at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:209) at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:181) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:208) at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:137) at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:178) at org.apache.hive.beeline.Commands.close(Commands.java:1173) at org.apache.hive.beeline.Commands.closeall(Commands.java:1155) at org.apache.hive.beeline.BeeLine.close(BeeLine.java:930) at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:847) at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:482) at org.apache.hive.beeline.BeeLine.main(BeeLine.java:465) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7)) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:770) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) ... 28 more Caused by: KrbException: Server not found in Kerberos database (7) at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:70) at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:259) at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:270) at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:302) at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:120) at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:458) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:693) ... 31 more Caused by: KrbException: Identifier doesn't match expected value (906) at sun.security.krb5.internal.KDCRep.init(KDCRep.java:140) at sun.security.krb5.internal.TGSRep.init(TGSRep.java:65) at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:60) at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55) ... 37 more Error: Could not open client transport with JDBC Uri: jdbc:hive2://vjlb:10000/default;principal=hive/_h...@gce.cloudera.com: GSS initiate failed (state=08S01,code=0) [root@vjd-1 ~]# {code} Please find the log excerpt with correct string with canonical name and canonical functionality turned off in connect string. {code} [root@vjd-1 ~]# beeline -u "jdbc:hive2://vjtest-local.gce.cloudera.com:10000/default;principal=hive/_h...@gce.cloudera.com" -e "show tables;" Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0 Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release 16/05/22 01:22:11 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree module jar containing PrefixTreeCodec is not present. Continuing without it. Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0 scan complete in 2ms Connecting to jdbc:hive2://vjtest-local.gce.cloudera.com:10000/default;principal=hive/_h...@gce.cloudera.com Connected to: Apache Hive (version 1.1.0-cdh5.7.0) Driver: Hive JDBC (version 1.1.0-cdh5.7.0) Transaction isolation: TRANSACTION_REPEATABLE_READ INFO : Compiling command(queryId=hive_20160522012222_124ecbe1-27f8-4bd4-984a-4701b2cd0fe9): show tables INFO : Semantic Analysis Completed INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null) INFO : Completed compiling command(queryId=hive_20160522012222_124ecbe1-27f8-4bd4-984a-4701b2cd0fe9); Time taken: 0.083 seconds INFO : Executing command(queryId=hive_20160522012222_124ecbe1-27f8-4bd4-984a-4701b2cd0fe9): show tables INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20160522012222_124ecbe1-27f8-4bd4-984a-4701b2cd0fe9); Time taken: 0.142 seconds INFO : OK +------------+--+ | tab_name | +------------+--+ | customers | | sample_07 | | sample_08 | | web_logs | +------------+--+ 4 rows selected (0.385 seconds) Beeline version 1.1.0-cdh5.7.0 by Apache Hive Closing: 0: jdbc:hive2://vjtest-local.gce.cloudera.com:10000/default;principal=hive/_h...@gce.cloudera.com [root@vjd-1 ~]# {code} Please review and provide feedback. > Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow > for failover > ---------------------------------------------------------------------------------------- > > Key: HIVE-13817 > URL: https://issues.apache.org/jira/browse/HIVE-13817 > Project: Hive > Issue Type: New Feature > Components: Beeline > Affects Versions: 1.2.1 > Reporter: Vijay Singh > Attachments: HIVE-13817.1.patch, HIVE-13817.2.patch, > HIVE-13817.3.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > Currently, in case of BDR clusters, DNS CNAME alias based connections fail. > As _HOST resolves to exact endpoint specified in connection string and that > may not be intended SPN for kerberos based on reverse DNS lookup. > Consequently this JIRA proposes that client specific setting be used to > resolv _HOST from CNAME DNS alias to A record entry on the fly in beeline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)