[jira] [Commented] (HIVE-17368) DBTokenStore fails to connect in Kerberos enabled remote HMS environment
[ https://issues.apache.org/jira/browse/HIVE-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16214492#comment-16214492 ] Vihang Karajgaonkar commented on HIVE-17368: Patch merged to master > DBTokenStore fails to connect in Kerberos enabled remote HMS environment > > > Key: HIVE-17368 > URL: https://issues.apache.org/jira/browse/HIVE-17368 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0, 2.1.0, 2.2.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Fix For: 3.0.0, 2.4.0 > > Attachments: HIVE-17368.01-branch-2.patch, HIVE-17368.01.patch, > HIVE-17368.02-branch-2.patch, HIVE-17368.02.patch, > HIVE-17368.03-branch-2.patch, HIVE-17368.04-branch-2.patch, > HIVE-17368.05-branch-2.patch, HIVE-17368.06-branch-2.patch > > > In setups where HMS is running as a remote process secured using Kerberos, > and when {{DBTokenStore}} is configured as the token store, the HS2 Thrift > API call {{GetDelegationToken}} fail with exception trace seen below. HS2 is > not able to invoke HMS APIs needed to add/remove/renew tokens from the DB > since it is possible that the user which is issue the {{GetDelegationToken}} > is not kerberos enabled. > Eg. Oozie submits a job on behalf of user "Joe". When Oozie opens a session > with HS2 it uses Oozie's principal and creates a proxy UGI with Hive. This > principal can establish a transport authenticated using Kerberos. It stores > the HMS delegation token string in the sessionConf and sessionToken. Now, > lets say Oozie issues a {{GetDelegationToken}} which has {{Joe}} as the owner > and {{oozie}} as the renewer in {{GetDelegationTokenReq}}. This API call > cannot instantiate a HMSClient and open transport to HMS using the HMSToken > string available in the sessionConf, since DBTokenStore uses server HiveConf > instead of sessionConf. It tries to establish transport using Kerberos and it > fails since user Joe is not Kerberos enabled. > I see the following exception trace in HS2 logs. > {noformat} > 2017-08-21T18:07:19,644 ERROR [HiveServer2-Handler-Pool: Thread-61] > transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) > ~[?:1.8.0_121] > at > org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at java.security.AccessController.doPrivileged(Native Method) > ~[?:1.8.0_121] > at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_121] > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > [hadoop-common-2.7.2.jar:?] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:488) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:255) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:70) > [hive-exec-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) ~[?:1.8.0_121] > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > [?:1.8.0_121] > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > [?:1.8.0_121] > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > [?:1.8.0_121] > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1699) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:83) >
[jira] [Commented] (HIVE-17368) DBTokenStore fails to connect in Kerberos enabled remote HMS environment
[ https://issues.apache.org/jira/browse/HIVE-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16214485#comment-16214485 ] Vihang Karajgaonkar commented on HIVE-17368: test failures are unrelated. > DBTokenStore fails to connect in Kerberos enabled remote HMS environment > > > Key: HIVE-17368 > URL: https://issues.apache.org/jira/browse/HIVE-17368 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0, 2.1.0, 2.2.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Attachments: HIVE-17368.01-branch-2.patch, HIVE-17368.01.patch, > HIVE-17368.02-branch-2.patch, HIVE-17368.02.patch, > HIVE-17368.03-branch-2.patch, HIVE-17368.04-branch-2.patch, > HIVE-17368.05-branch-2.patch, HIVE-17368.06-branch-2.patch > > > In setups where HMS is running as a remote process secured using Kerberos, > and when {{DBTokenStore}} is configured as the token store, the HS2 Thrift > API call {{GetDelegationToken}} fail with exception trace seen below. HS2 is > not able to invoke HMS APIs needed to add/remove/renew tokens from the DB > since it is possible that the user which is issue the {{GetDelegationToken}} > is not kerberos enabled. > Eg. Oozie submits a job on behalf of user "Joe". When Oozie opens a session > with HS2 it uses Oozie's principal and creates a proxy UGI with Hive. This > principal can establish a transport authenticated using Kerberos. It stores > the HMS delegation token string in the sessionConf and sessionToken. Now, > lets say Oozie issues a {{GetDelegationToken}} which has {{Joe}} as the owner > and {{oozie}} as the renewer in {{GetDelegationTokenReq}}. This API call > cannot instantiate a HMSClient and open transport to HMS using the HMSToken > string available in the sessionConf, since DBTokenStore uses server HiveConf > instead of sessionConf. It tries to establish transport using Kerberos and it > fails since user Joe is not Kerberos enabled. > I see the following exception trace in HS2 logs. > {noformat} > 2017-08-21T18:07:19,644 ERROR [HiveServer2-Handler-Pool: Thread-61] > transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) > ~[?:1.8.0_121] > at > org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at java.security.AccessController.doPrivileged(Native Method) > ~[?:1.8.0_121] > at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_121] > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > [hadoop-common-2.7.2.jar:?] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:488) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:255) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:70) > [hive-exec-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) ~[?:1.8.0_121] > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > [?:1.8.0_121] > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > [?:1.8.0_121] > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > [?:1.8.0_121] > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1699) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:83) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at >
[jira] [Commented] (HIVE-17368) DBTokenStore fails to connect in Kerberos enabled remote HMS environment
[ https://issues.apache.org/jira/browse/HIVE-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16213227#comment-16213227 ] Hive QA commented on HIVE-17368: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12893313/HIVE-17368.02.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 11314 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2] (batchId=47) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast] (batchId=156) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan] (batchId=163) org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut (batchId=204) org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints (batchId=221) org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes (batchId=228) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7413/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7413/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7413/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 6 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12893313 - PreCommit-HIVE-Build > DBTokenStore fails to connect in Kerberos enabled remote HMS environment > > > Key: HIVE-17368 > URL: https://issues.apache.org/jira/browse/HIVE-17368 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0, 2.1.0, 2.2.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Attachments: HIVE-17368.01-branch-2.patch, HIVE-17368.01.patch, > HIVE-17368.02-branch-2.patch, HIVE-17368.02.patch, > HIVE-17368.03-branch-2.patch, HIVE-17368.04-branch-2.patch, > HIVE-17368.05-branch-2.patch, HIVE-17368.06-branch-2.patch > > > In setups where HMS is running as a remote process secured using Kerberos, > and when {{DBTokenStore}} is configured as the token store, the HS2 Thrift > API call {{GetDelegationToken}} fail with exception trace seen below. HS2 is > not able to invoke HMS APIs needed to add/remove/renew tokens from the DB > since it is possible that the user which is issue the {{GetDelegationToken}} > is not kerberos enabled. > Eg. Oozie submits a job on behalf of user "Joe". When Oozie opens a session > with HS2 it uses Oozie's principal and creates a proxy UGI with Hive. This > principal can establish a transport authenticated using Kerberos. It stores > the HMS delegation token string in the sessionConf and sessionToken. Now, > lets say Oozie issues a {{GetDelegationToken}} which has {{Joe}} as the owner > and {{oozie}} as the renewer in {{GetDelegationTokenReq}}. This API call > cannot instantiate a HMSClient and open transport to HMS using the HMSToken > string available in the sessionConf, since DBTokenStore uses server HiveConf > instead of sessionConf. It tries to establish transport using Kerberos and it > fails since user Joe is not Kerberos enabled. > I see the following exception trace in HS2 logs. > {noformat} > 2017-08-21T18:07:19,644 ERROR [HiveServer2-Handler-Pool: Thread-61] > transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) > ~[?:1.8.0_121] > at > org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at java.security.AccessController.doPrivileged(Native Method) > ~[?:1.8.0_121] > at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_121] > at >
[jira] [Commented] (HIVE-17368) DBTokenStore fails to connect in Kerberos enabled remote HMS environment
[ https://issues.apache.org/jira/browse/HIVE-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155514#comment-16155514 ] Vihang Karajgaonkar commented on HIVE-17368: I would merge the patch to master once HIVE-17371 is fixed. Keeping this open until then. > DBTokenStore fails to connect in Kerberos enabled remote HMS environment > > > Key: HIVE-17368 > URL: https://issues.apache.org/jira/browse/HIVE-17368 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0, 2.1.0, 2.2.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Attachments: HIVE-17368.01-branch-2.patch, HIVE-17368.01.patch, > HIVE-17368.02-branch-2.patch, HIVE-17368.03-branch-2.patch, > HIVE-17368.04-branch-2.patch, HIVE-17368.05-branch-2.patch, > HIVE-17368.06-branch-2.patch > > > In setups where HMS is running as a remote process secured using Kerberos, > and when {{DBTokenStore}} is configured as the token store, the HS2 Thrift > API calls like {{GetDelegationToken}}, {{CancelDelegationToken}} and > {{RenewDelegationToken}} fail with exception trace seen below. HS2 is not > able to invoke HMS APIs needed to add/remove/renew tokens from the DB since > it is possible that the user which is issue the {{GetDelegationToken}} is not > kerberos enabled. > Eg. Oozie submits a job on behalf of user "Joe". When Oozie opens a session > with HS2 it uses Oozie's principal and creates a proxy UGI with Hive. This > principal can establish a transport authenticated using Kerberos. It stores > the HMS delegation token string in the sessionConf and sessionToken. Now, > lets say Oozie issues a {{GetDelegationToken}} which has {{Joe}} as the owner > and {{oozie}} as the renewer in {{GetDelegationTokenReq}}. This API call > cannot instantiate a HMSClient and open transport to HMS using the HMSToken > string available in the sessionConf, since DBTokenStore uses server HiveConf > instead of sessionConf. It tries to establish transport using Kerberos and it > fails since user Joe is not Kerberos enabled. > I see the following exception trace in HS2 logs. > {noformat} > 2017-08-21T18:07:19,644 ERROR [HiveServer2-Handler-Pool: Thread-61] > transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) > ~[?:1.8.0_121] > at > org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at java.security.AccessController.doPrivileged(Native Method) > ~[?:1.8.0_121] > at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_121] > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > [hadoop-common-2.7.2.jar:?] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:488) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:255) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:70) > [hive-exec-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) ~[?:1.8.0_121] > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > [?:1.8.0_121] > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > [?:1.8.0_121] > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > [?:1.8.0_121] > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1699) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at >
[jira] [Commented] (HIVE-17368) DBTokenStore fails to connect in Kerberos enabled remote HMS environment
[ https://issues.apache.org/jira/browse/HIVE-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155511#comment-16155511 ] Vihang Karajgaonkar commented on HIVE-17368: Merged to branch-2. Thanks [~aihuaxu] and [~janulatha] for the review. > DBTokenStore fails to connect in Kerberos enabled remote HMS environment > > > Key: HIVE-17368 > URL: https://issues.apache.org/jira/browse/HIVE-17368 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0, 2.1.0, 2.2.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Attachments: HIVE-17368.01-branch-2.patch, HIVE-17368.01.patch, > HIVE-17368.02-branch-2.patch, HIVE-17368.03-branch-2.patch, > HIVE-17368.04-branch-2.patch, HIVE-17368.05-branch-2.patch, > HIVE-17368.06-branch-2.patch > > > In setups where HMS is running as a remote process secured using Kerberos, > and when {{DBTokenStore}} is configured as the token store, the HS2 Thrift > API calls like {{GetDelegationToken}}, {{CancelDelegationToken}} and > {{RenewDelegationToken}} fail with exception trace seen below. HS2 is not > able to invoke HMS APIs needed to add/remove/renew tokens from the DB since > it is possible that the user which is issue the {{GetDelegationToken}} is not > kerberos enabled. > Eg. Oozie submits a job on behalf of user "Joe". When Oozie opens a session > with HS2 it uses Oozie's principal and creates a proxy UGI with Hive. This > principal can establish a transport authenticated using Kerberos. It stores > the HMS delegation token string in the sessionConf and sessionToken. Now, > lets say Oozie issues a {{GetDelegationToken}} which has {{Joe}} as the owner > and {{oozie}} as the renewer in {{GetDelegationTokenReq}}. This API call > cannot instantiate a HMSClient and open transport to HMS using the HMSToken > string available in the sessionConf, since DBTokenStore uses server HiveConf > instead of sessionConf. It tries to establish transport using Kerberos and it > fails since user Joe is not Kerberos enabled. > I see the following exception trace in HS2 logs. > {noformat} > 2017-08-21T18:07:19,644 ERROR [HiveServer2-Handler-Pool: Thread-61] > transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) > ~[?:1.8.0_121] > at > org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at java.security.AccessController.doPrivileged(Native Method) > ~[?:1.8.0_121] > at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_121] > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > [hadoop-common-2.7.2.jar:?] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:488) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:255) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:70) > [hive-exec-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) ~[?:1.8.0_121] > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > [?:1.8.0_121] > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > [?:1.8.0_121] > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > [?:1.8.0_121] > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1699) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at >
[jira] [Commented] (HIVE-17368) DBTokenStore fails to connect in Kerberos enabled remote HMS environment
[ https://issues.apache.org/jira/browse/HIVE-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155504#comment-16155504 ] Vihang Karajgaonkar commented on HIVE-17368: Test failures are unrelated > DBTokenStore fails to connect in Kerberos enabled remote HMS environment > > > Key: HIVE-17368 > URL: https://issues.apache.org/jira/browse/HIVE-17368 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0, 2.1.0, 2.2.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Attachments: HIVE-17368.01-branch-2.patch, HIVE-17368.01.patch, > HIVE-17368.02-branch-2.patch, HIVE-17368.03-branch-2.patch, > HIVE-17368.04-branch-2.patch, HIVE-17368.05-branch-2.patch, > HIVE-17368.06-branch-2.patch > > > In setups where HMS is running as a remote process secured using Kerberos, > and when {{DBTokenStore}} is configured as the token store, the HS2 Thrift > API calls like {{GetDelegationToken}}, {{CancelDelegationToken}} and > {{RenewDelegationToken}} fail with exception trace seen below. HS2 is not > able to invoke HMS APIs needed to add/remove/renew tokens from the DB since > it is possible that the user which is issue the {{GetDelegationToken}} is not > kerberos enabled. > Eg. Oozie submits a job on behalf of user "Joe". When Oozie opens a session > with HS2 it uses Oozie's principal and creates a proxy UGI with Hive. This > principal can establish a transport authenticated using Kerberos. It stores > the HMS delegation token string in the sessionConf and sessionToken. Now, > lets say Oozie issues a {{GetDelegationToken}} which has {{Joe}} as the owner > and {{oozie}} as the renewer in {{GetDelegationTokenReq}}. This API call > cannot instantiate a HMSClient and open transport to HMS using the HMSToken > string available in the sessionConf, since DBTokenStore uses server HiveConf > instead of sessionConf. It tries to establish transport using Kerberos and it > fails since user Joe is not Kerberos enabled. > I see the following exception trace in HS2 logs. > {noformat} > 2017-08-21T18:07:19,644 ERROR [HiveServer2-Handler-Pool: Thread-61] > transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) > ~[?:1.8.0_121] > at > org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at java.security.AccessController.doPrivileged(Native Method) > ~[?:1.8.0_121] > at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_121] > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > [hadoop-common-2.7.2.jar:?] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:488) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:255) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:70) > [hive-exec-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) ~[?:1.8.0_121] > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > [?:1.8.0_121] > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > [?:1.8.0_121] > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > [?:1.8.0_121] > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1699) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:83) >
[jira] [Commented] (HIVE-17368) DBTokenStore fails to connect in Kerberos enabled remote HMS environment
[ https://issues.apache.org/jira/browse/HIVE-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154915#comment-16154915 ] Hive QA commented on HIVE-17368: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12885478/HIVE-17368.06-branch-2.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10589 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explaindenpendencydiffengs] (batchId=38) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] (batchId=142) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic] (batchId=139) org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver (batchId=104) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs] (batchId=115) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_ptf] (batchId=125) org.apache.hadoop.hive.ql.security.TestExtendedAcls.testPartition (batchId=228) org.apache.hadoop.hive.ql.security.TestFolderPermissions.testPartition (batchId=217) org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=176) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6685/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6685/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6685/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 10 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12885478 - PreCommit-HIVE-Build > DBTokenStore fails to connect in Kerberos enabled remote HMS environment > > > Key: HIVE-17368 > URL: https://issues.apache.org/jira/browse/HIVE-17368 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0, 2.1.0, 2.2.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Attachments: HIVE-17368.01-branch-2.patch, HIVE-17368.01.patch, > HIVE-17368.02-branch-2.patch, HIVE-17368.03-branch-2.patch, > HIVE-17368.04-branch-2.patch, HIVE-17368.05-branch-2.patch, > HIVE-17368.06-branch-2.patch > > > In setups where HMS is running as a remote process secured using Kerberos, > and when {{DBTokenStore}} is configured as the token store, the HS2 Thrift > API calls like {{GetDelegationToken}}, {{CancelDelegationToken}} and > {{RenewDelegationToken}} fail with exception trace seen below. HS2 is not > able to invoke HMS APIs needed to add/remove/renew tokens from the DB since > it is possible that the user which is issue the {{GetDelegationToken}} is not > kerberos enabled. > Eg. Oozie submits a job on behalf of user "Joe". When Oozie opens a session > with HS2 it uses Oozie's principal and creates a proxy UGI with Hive. This > principal can establish a transport authenticated using Kerberos. It stores > the HMS delegation token string in the sessionConf and sessionToken. Now, > lets say Oozie issues a {{GetDelegationToken}} which has {{Joe}} as the owner > and {{oozie}} as the renewer in {{GetDelegationTokenReq}}. This API call > cannot instantiate a HMSClient and open transport to HMS using the HMSToken > string available in the sessionConf, since DBTokenStore uses server HiveConf > instead of sessionConf. It tries to establish transport using Kerberos and it > fails since user Joe is not Kerberos enabled. > I see the following exception trace in HS2 logs. > {noformat} > 2017-08-21T18:07:19,644 ERROR [HiveServer2-Handler-Pool: Thread-61] > transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) > ~[?:1.8.0_121] > at > org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at >
[jira] [Commented] (HIVE-17368) DBTokenStore fails to connect in Kerberos enabled remote HMS environment
[ https://issues.apache.org/jira/browse/HIVE-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154860#comment-16154860 ] Hive QA commented on HIVE-17368: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12885478/HIVE-17368.06-branch-2.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 10603 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explaindenpendencydiffengs] (batchId=38) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] (batchId=142) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic] (batchId=139) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs] (batchId=115) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_ptf] (batchId=125) org.apache.hadoop.hive.ql.security.TestExtendedAcls.testPartition (batchId=228) org.apache.hadoop.hive.ql.security.TestFolderPermissions.testPartition (batchId=217) org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=176) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6684/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6684/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6684/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 9 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12885478 - PreCommit-HIVE-Build > DBTokenStore fails to connect in Kerberos enabled remote HMS environment > > > Key: HIVE-17368 > URL: https://issues.apache.org/jira/browse/HIVE-17368 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0, 2.1.0, 2.2.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Attachments: HIVE-17368.01-branch-2.patch, HIVE-17368.01.patch, > HIVE-17368.02-branch-2.patch, HIVE-17368.03-branch-2.patch, > HIVE-17368.04-branch-2.patch, HIVE-17368.05-branch-2.patch, > HIVE-17368.06-branch-2.patch > > > In setups where HMS is running as a remote process secured using Kerberos, > and when {{DBTokenStore}} is configured as the token store, the HS2 Thrift > API calls like {{GetDelegationToken}}, {{CancelDelegationToken}} and > {{RenewDelegationToken}} fail with exception trace seen below. HS2 is not > able to invoke HMS APIs needed to add/remove/renew tokens from the DB since > it is possible that the user which is issue the {{GetDelegationToken}} is not > kerberos enabled. > Eg. Oozie submits a job on behalf of user "Joe". When Oozie opens a session > with HS2 it uses Oozie's principal and creates a proxy UGI with Hive. This > principal can establish a transport authenticated using Kerberos. It stores > the HMS delegation token string in the sessionConf and sessionToken. Now, > lets say Oozie issues a {{GetDelegationToken}} which has {{Joe}} as the owner > and {{oozie}} as the renewer in {{GetDelegationTokenReq}}. This API call > cannot instantiate a HMSClient and open transport to HMS using the HMSToken > string available in the sessionConf, since DBTokenStore uses server HiveConf > instead of sessionConf. It tries to establish transport using Kerberos and it > fails since user Joe is not Kerberos enabled. > I see the following exception trace in HS2 logs. > {noformat} > 2017-08-21T18:07:19,644 ERROR [HiveServer2-Handler-Pool: Thread-61] > transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) > ~[?:1.8.0_121] > at > org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) >
[jira] [Commented] (HIVE-17368) DBTokenStore fails to connect in Kerberos enabled remote HMS environment
[ https://issues.apache.org/jira/browse/HIVE-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154139#comment-16154139 ] Aihua Xu commented on HIVE-17368: - That makes sense. The change looks good to me. +1. > DBTokenStore fails to connect in Kerberos enabled remote HMS environment > > > Key: HIVE-17368 > URL: https://issues.apache.org/jira/browse/HIVE-17368 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0, 2.1.0, 2.2.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Attachments: HIVE-17368.01-branch-2.patch, HIVE-17368.01.patch, > HIVE-17368.02-branch-2.patch, HIVE-17368.03-branch-2.patch, > HIVE-17368.04-branch-2.patch, HIVE-17368.05-branch-2.patch > > > In setups where HMS is running as a remote process secured using Kerberos, > and when {{DBTokenStore}} is configured as the token store, the HS2 Thrift > API calls like {{GetDelegationToken}}, {{CancelDelegationToken}} and > {{RenewDelegationToken}} fail with exception trace seen below. HS2 is not > able to invoke HMS APIs needed to add/remove/renew tokens from the DB since > it is possible that the user which is issue the {{GetDelegationToken}} is not > kerberos enabled. > Eg. Oozie submits a job on behalf of user "Joe". When Oozie opens a session > with HS2 it uses Oozie's principal and creates a proxy UGI with Hive. This > principal can establish a transport authenticated using Kerberos. It stores > the HMS delegation token string in the sessionConf and sessionToken. Now, > lets say Oozie issues a {{GetDelegationToken}} which has {{Joe}} as the owner > and {{oozie}} as the renewer in {{GetDelegationTokenReq}}. This API call > cannot instantiate a HMSClient and open transport to HMS using the HMSToken > string available in the sessionConf, since DBTokenStore uses server HiveConf > instead of sessionConf. It tries to establish transport using Kerberos and it > fails since user Joe is not Kerberos enabled. > I see the following exception trace in HS2 logs. > {noformat} > 2017-08-21T18:07:19,644 ERROR [HiveServer2-Handler-Pool: Thread-61] > transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) > ~[?:1.8.0_121] > at > org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at java.security.AccessController.doPrivileged(Native Method) > ~[?:1.8.0_121] > at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_121] > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > [hadoop-common-2.7.2.jar:?] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:488) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:255) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:70) > [hive-exec-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) ~[?:1.8.0_121] > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > [?:1.8.0_121] > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > [?:1.8.0_121] > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > [?:1.8.0_121] > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1699) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:83) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] >
[jira] [Commented] (HIVE-17368) DBTokenStore fails to connect in Kerberos enabled remote HMS environment
[ https://issues.apache.org/jira/browse/HIVE-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154138#comment-16154138 ] Vihang Karajgaonkar commented on HIVE-17368: added https://reviews.apache.org/r/62092/ [~aihuaxu] I changed to {{.getMethod("get").invoke(handler, null);}} so that it uses the {{Hive.get()}} which uses sessionconf instead of the existing implementation which uses service configuration object. The Session configuration object is required for the call when impersonation is turned on because HMSDelegationToken is stored in the sessionConf not the service conf. > DBTokenStore fails to connect in Kerberos enabled remote HMS environment > > > Key: HIVE-17368 > URL: https://issues.apache.org/jira/browse/HIVE-17368 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0, 2.1.0, 2.2.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Attachments: HIVE-17368.01-branch-2.patch, HIVE-17368.01.patch, > HIVE-17368.02-branch-2.patch, HIVE-17368.03-branch-2.patch, > HIVE-17368.04-branch-2.patch, HIVE-17368.05-branch-2.patch > > > In setups where HMS is running as a remote process secured using Kerberos, > and when {{DBTokenStore}} is configured as the token store, the HS2 Thrift > API calls like {{GetDelegationToken}}, {{CancelDelegationToken}} and > {{RenewDelegationToken}} fail with exception trace seen below. HS2 is not > able to invoke HMS APIs needed to add/remove/renew tokens from the DB since > it is possible that the user which is issue the {{GetDelegationToken}} is not > kerberos enabled. > Eg. Oozie submits a job on behalf of user "Joe". When Oozie opens a session > with HS2 it uses Oozie's principal and creates a proxy UGI with Hive. This > principal can establish a transport authenticated using Kerberos. It stores > the HMS delegation token string in the sessionConf and sessionToken. Now, > lets say Oozie issues a {{GetDelegationToken}} which has {{Joe}} as the owner > and {{oozie}} as the renewer in {{GetDelegationTokenReq}}. This API call > cannot instantiate a HMSClient and open transport to HMS using the HMSToken > string available in the sessionConf, since DBTokenStore uses server HiveConf > instead of sessionConf. It tries to establish transport using Kerberos and it > fails since user Joe is not Kerberos enabled. > I see the following exception trace in HS2 logs. > {noformat} > 2017-08-21T18:07:19,644 ERROR [HiveServer2-Handler-Pool: Thread-61] > transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) > ~[?:1.8.0_121] > at > org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at java.security.AccessController.doPrivileged(Native Method) > ~[?:1.8.0_121] > at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_121] > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > [hadoop-common-2.7.2.jar:?] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:488) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:255) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:70) > [hive-exec-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) ~[?:1.8.0_121] > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > [?:1.8.0_121] > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > [?:1.8.0_121] > at
[jira] [Commented] (HIVE-17368) DBTokenStore fails to connect in Kerberos enabled remote HMS environment
[ https://issues.apache.org/jira/browse/HIVE-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154093#comment-16154093 ] Aihua Xu commented on HIVE-17368: - [~vihangk1] Can you create a RB for the change? What's the reason to change to {{.getMethod("get").invoke(handler, null);}} > DBTokenStore fails to connect in Kerberos enabled remote HMS environment > > > Key: HIVE-17368 > URL: https://issues.apache.org/jira/browse/HIVE-17368 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0, 2.1.0, 2.2.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Attachments: HIVE-17368.01-branch-2.patch, HIVE-17368.01.patch, > HIVE-17368.02-branch-2.patch, HIVE-17368.03-branch-2.patch, > HIVE-17368.04-branch-2.patch, HIVE-17368.05-branch-2.patch > > > In setups where HMS is running as a remote process secured using Kerberos, > and when {{DBTokenStore}} is configured as the token store, the HS2 Thrift > API calls like {{GetDelegationToken}}, {{CancelDelegationToken}} and > {{RenewDelegationToken}} fail with exception trace seen below. HS2 is not > able to invoke HMS APIs needed to add/remove/renew tokens from the DB since > it is possible that the user which is issue the {{GetDelegationToken}} is not > kerberos enabled. > Eg. Oozie submits a job on behalf of user "Joe". When Oozie opens a session > with HS2 it uses Oozie's principal and creates a proxy UGI with Hive. This > principal can establish a transport authenticated using Kerberos. It stores > the HMS delegation token string in the sessionConf and sessionToken. Now, > lets say Oozie issues a {{GetDelegationToken}} which has {{Joe}} as the owner > and {{oozie}} as the renewer in {{GetDelegationTokenReq}}. This API call > cannot instantiate a HMSClient and open transport to HMS using the HMSToken > string available in the sessionConf, since DBTokenStore uses server HiveConf > instead of sessionConf. It tries to establish transport using Kerberos and it > fails since user Joe is not Kerberos enabled. > I see the following exception trace in HS2 logs. > {noformat} > 2017-08-21T18:07:19,644 ERROR [HiveServer2-Handler-Pool: Thread-61] > transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) > ~[?:1.8.0_121] > at > org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at java.security.AccessController.doPrivileged(Native Method) > ~[?:1.8.0_121] > at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_121] > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > [hadoop-common-2.7.2.jar:?] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:488) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:255) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:70) > [hive-exec-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) ~[?:1.8.0_121] > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > [?:1.8.0_121] > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > [?:1.8.0_121] > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > [?:1.8.0_121] > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1699) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at >
[jira] [Commented] (HIVE-17368) DBTokenStore fails to connect in Kerberos enabled remote HMS environment
[ https://issues.apache.org/jira/browse/HIVE-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16152093#comment-16152093 ] Vihang Karajgaonkar commented on HIVE-17368: [~aihuaxu] [~thejas] Can you please review? > DBTokenStore fails to connect in Kerberos enabled remote HMS environment > > > Key: HIVE-17368 > URL: https://issues.apache.org/jira/browse/HIVE-17368 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0, 2.1.0, 2.2.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Attachments: HIVE-17368.01-branch-2.patch, HIVE-17368.01.patch, > HIVE-17368.02-branch-2.patch, HIVE-17368.03-branch-2.patch, > HIVE-17368.04-branch-2.patch, HIVE-17368.05-branch-2.patch > > > In setups where HMS is running as a remote process secured using Kerberos, > and when {{DBTokenStore}} is configured as the token store, the HS2 Thrift > API calls like {{GetDelegationToken}}, {{CancelDelegationToken}} and > {{RenewDelegationToken}} fail with exception trace seen below. HS2 is not > able to invoke HMS APIs needed to add/remove/renew tokens from the DB since > it is possible that the user which is issue the {{GetDelegationToken}} is not > kerberos enabled. > Eg. Oozie submits a job on behalf of user "Joe". When Oozie opens a session > with HS2 it uses Oozie's principal and creates a proxy UGI with Hive. This > principal can establish a transport authenticated using Kerberos. It stores > the HMS delegation token string in the sessionConf and sessionToken. Now, > lets say Oozie issues a {{GetDelegationToken}} which has {{Joe}} as the owner > and {{oozie}} as the renewer in {{GetDelegationTokenReq}}. This API call > cannot instantiate a HMSClient and open transport to HMS using the HMSToken > string available in the sessionConf, since DBTokenStore uses server HiveConf > instead of sessionConf. It tries to establish transport using Kerberos and it > fails since user Joe is not Kerberos enabled. > I see the following exception trace in HS2 logs. > {noformat} > 2017-08-21T18:07:19,644 ERROR [HiveServer2-Handler-Pool: Thread-61] > transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) > ~[?:1.8.0_121] > at > org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at java.security.AccessController.doPrivileged(Native Method) > ~[?:1.8.0_121] > at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_121] > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > [hadoop-common-2.7.2.jar:?] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:488) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:255) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:70) > [hive-exec-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) ~[?:1.8.0_121] > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > [?:1.8.0_121] > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > [?:1.8.0_121] > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > [?:1.8.0_121] > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1699) > [hive-metastore-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:83) >
[jira] [Commented] (HIVE-17368) DBTokenStore fails to connect in Kerberos enabled remote HMS environment
[ https://issues.apache.org/jira/browse/HIVE-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16152024#comment-16152024 ] Hive QA commented on HIVE-17368: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12885172/HIVE-17368.05-branch-2.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 10603 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explaindenpendencydiffengs] (batchId=38) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] (batchId=142) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic] (batchId=139) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs] (batchId=115) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_ptf] (batchId=125) org.apache.hadoop.hive.ql.security.TestExtendedAcls.testPartition (batchId=228) org.apache.hadoop.hive.ql.security.TestFolderPermissions.testPartition (batchId=217) org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=176) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6664/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6664/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6664/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 9 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12885172 - PreCommit-HIVE-Build > DBTokenStore fails to connect in Kerberos enabled remote HMS environment > > > Key: HIVE-17368 > URL: https://issues.apache.org/jira/browse/HIVE-17368 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0, 2.1.0, 2.2.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Attachments: HIVE-17368.01-branch-2.patch, HIVE-17368.01.patch, > HIVE-17368.02-branch-2.patch, HIVE-17368.03-branch-2.patch, > HIVE-17368.04-branch-2.patch, HIVE-17368.05-branch-2.patch > > > In setups where HMS is running as a remote process secured using Kerberos, > and when {{DBTokenStore}} is configured as the token store, the HS2 Thrift > API calls like {{GetDelegationToken}}, {{CancelDelegationToken}} and > {{RenewDelegationToken}} fail with exception trace seen below. HS2 is not > able to invoke HMS APIs needed to add/remove/renew tokens from the DB since > it is possible that the user which is issue the {{GetDelegationToken}} is not > kerberos enabled. > Eg. Oozie submits a job on behalf of user "Joe". When Oozie opens a session > with HS2 it uses Oozie's principal and creates a proxy UGI with Hive. This > principal can establish a transport authenticated using Kerberos. It stores > the HMS delegation token string in the sessionConf and sessionToken. Now, > lets say Oozie issues a {{GetDelegationToken}} which has {{Joe}} as the owner > and {{oozie}} as the renewer in {{GetDelegationTokenReq}}. This API call > cannot instantiate a HMSClient and open transport to HMS using the HMSToken > string available in the sessionConf, since DBTokenStore uses server HiveConf > instead of sessionConf. It tries to establish transport using Kerberos and it > fails since user Joe is not Kerberos enabled. > I see the following exception trace in HS2 logs. > {noformat} > 2017-08-21T18:07:19,644 ERROR [HiveServer2-Handler-Pool: Thread-61] > transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) > ~[?:1.8.0_121] > at > org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] >
[jira] [Commented] (HIVE-17368) DBTokenStore fails to connect in Kerberos enabled remote HMS environment
[ https://issues.apache.org/jira/browse/HIVE-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151570#comment-16151570 ] Hive QA commented on HIVE-17368: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12885095/HIVE-17368.04-branch-2.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 10603 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explaindenpendencydiffengs] (batchId=38) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] (batchId=142) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic] (batchId=139) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype] (batchId=156) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=144) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs] (batchId=115) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_ptf] (batchId=125) org.apache.hadoop.hive.ql.security.TestExtendedAcls.testPartition (batchId=228) org.apache.hadoop.hive.ql.security.TestFolderPermissions.testPartition (batchId=217) org.apache.hadoop.hive.thrift.TestHadoopAuthBridge23.testMetastoreProxyUser (batchId=228) org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=176) org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNegativeTokenAuth (batchId=237) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeTokenAuth (batchId=237) org.apache.hive.minikdc.TestJdbcWithMiniKdc.testNegativeTokenAuth (batchId=237) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6657/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6657/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6657/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 15 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12885095 - PreCommit-HIVE-Build > DBTokenStore fails to connect in Kerberos enabled remote HMS environment > > > Key: HIVE-17368 > URL: https://issues.apache.org/jira/browse/HIVE-17368 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0, 2.1.0, 2.2.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Attachments: HIVE-17368.01-branch-2.patch, HIVE-17368.01.patch, > HIVE-17368.02-branch-2.patch, HIVE-17368.03-branch-2.patch, > HIVE-17368.04-branch-2.patch > > > In setups where HMS is running as a remote process secured using Kerberos, > and when {{DBTokenStore}} is configured as the token store, the HS2 Thrift > API calls like {{GetDelegationToken}}, {{CancelDelegationToken}} and > {{RenewDelegationToken}} fail with exception trace seen below. HS2 is not > able to invoke HMS APIs needed to add/remove/renew tokens from the DB since > it is possible that the user which is issue the {{GetDelegationToken}} is not > kerberos enabled. > Eg. Oozie submits a job on behalf of user "Joe". When Oozie opens a session > with HS2 it uses Oozie's principal and creates a proxy UGI with Hive. This > principal can establish a transport authenticated using Kerberos. It stores > the HMS delegation token string in the sessionConf and sessionToken. Now, > lets say Oozie issues a {{GetDelegationToken}} which has {{Joe}} as the owner > and {{oozie}} as the renewer in {{GetDelegationTokenReq}}. This API call > cannot instantiate a HMSClient and open transport to HMS using the HMSToken > string available in the sessionConf, since DBTokenStore uses server HiveConf > instead of sessionConf. It tries to establish transport using Kerberos and it > fails since user Joe is not Kerberos enabled. > I see the following exception trace in HS2 logs. > {noformat} > 2017-08-21T18:07:19,644 ERROR [HiveServer2-Handler-Pool: Thread-61] > transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) > ~[?:1.8.0_121] > at > org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[libthrift-0.9.3.jar:0.9.3] > at >
[jira] [Commented] (HIVE-17368) DBTokenStore fails to connect in Kerberos enabled remote HMS environment
[ https://issues.apache.org/jira/browse/HIVE-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151416#comment-16151416 ] Hive QA commented on HIVE-17368: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12885052/HIVE-17368.03-branch-2.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6654/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6654/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6654/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2017-09-02 05:39:27.367 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-6654/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z branch-2 ]] + [[ -d apache-github-branch-2-source ]] + [[ ! -d apache-github-branch-2-source/.git ]] + [[ ! -d apache-github-branch-2-source ]] + date '+%Y-%m-%d %T.%3N' 2017-09-02 05:39:27.370 + cd apache-github-branch-2-source + git fetch origin >From https://github.com/apache/hive 588148d..76933e7 branch-2 -> origin/branch-2 5a62503..714d7cf branch-2.1 -> origin/branch-2.1 120476d..b2e7d5e branch-2.2 -> origin/branch-2.2 6f4c35c..dee0a20 branch-2.3 -> origin/branch-2.3 6be50b7..d155565 master -> origin/master + git reset --hard HEAD HEAD is now at 588148d HIVE-17327 : ADDENDUM (revert a small part of the patch to fix the test) (Sergey Shelukhin) + git clean -f -d + git checkout branch-2 Already on 'branch-2' Your branch is behind 'origin/branch-2' by 2 commits, and can be fast-forwarded. (use "git pull" to update your local branch) + git reset --hard origin/branch-2 HEAD is now at 76933e7 HIVE-17411 : LLAP IO may incorrectly release a refcount in some rare cases (Sergey Shelukhin, reviewed by Prasanth Jayachandran) + git merge --ff-only origin/branch-2 Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2017-09-02 05:39:33.815 + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch Going to apply patch with: patch -p1 patching file itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/MiniHiveKdc.java patching file itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestJdbcWithDBTokenStore.java patching file itests/hive-unit-hadoop2/src/test/java/org/apache/hadoop/hive/thrift/TestHadoopAuthBridge23.java patching file itests/util/src/main/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java patching file ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java patching file service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java patching file shims/common/src/main/java/org/apache/hadoop/hive/thrift/DBTokenStore.java patching file shims/common/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java patching file shims/common/src/main/java/org/apache/hadoop/hive/thrift/HiveDelegationTokenManager.java + [[ maven == \m\a\v\e\n ]] + rm -rf /data/hiveptest/working/maven/org/apache/hive + mvn -B clean install -DskipTests -T 4 -q -Dmaven.repo.local=/data/hiveptest/working/maven [ERROR] COMPILATION ERROR : [ERROR] /data/hiveptest/working/apache-github-branch-2-source/shims/common/src/main/java/org/apache/hadoop/hive/thrift/HiveDelegationTokenManager.java:[124,25] cannot find symbol symbol: method getDSeelegationToken(java.lang.String,java.lang.String) location: variable secretManager of type org.apache.hadoop.hive.thrift.DelegationTokenSecretManager [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project hive-shims-common: Compilation failure [ERROR] /data/hiveptest/working/apache-github-branch-2-source/shims/common/src/main/java/org/apache/hadoop/hive/thrift/HiveDelegationTokenManager.java:[124,25]
[jira] [Commented] (HIVE-17368) DBTokenStore fails to connect in Kerberos enabled remote HMS environment
[ https://issues.apache.org/jira/browse/HIVE-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16142622#comment-16142622 ] Hive QA commented on HIVE-17368: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12883845/HIVE-17368.02-branch-2.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 10603 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explaindenpendencydiffengs] (batchId=38) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] (batchId=142) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic] (batchId=139) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs] (batchId=115) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_ptf] (batchId=125) org.apache.hadoop.hive.ql.security.TestExtendedAcls.testPartition (batchId=228) org.apache.hadoop.hive.ql.security.TestFolderPermissions.testPartition (batchId=217) org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=176) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6553/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6553/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6553/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 9 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12883845 - PreCommit-HIVE-Build > DBTokenStore fails to connect in Kerberos enabled remote HMS environment > > > Key: HIVE-17368 > URL: https://issues.apache.org/jira/browse/HIVE-17368 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0, 2.1.0, 2.2.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Attachments: HIVE-17368.01-branch-2.patch, HIVE-17368.01.patch, > HIVE-17368.02-branch-2.patch > > > In setups where HMS is running as a remote process secured using Kerberos, > and when {{DBTokenStore}} is configured as the token store, the HS2 Thrift > API calls like {{GetDelegationToken}}, {{CancelDelegationToken}} and > {{RenewDelegationToken}} fail with exception trace seen below. HS2 is not > able to invoke HMS APIs needed to add/remove/renew tokens from the DB since > it is possible that the user which is issue the {{GetDelegationToken}} is not > kerberos enabled. > Eg. Oozie submits a job on behalf of user "Joe". When Oozie opens a session > with HS2 it uses Oozie's principal and creates a proxy UGI with Hive. This > principal can establish a transport authenticated using Kerberos. It stores > the HMS delegation token string in the sessionConf and sessionToken. Now, > lets say Oozie issues a {{GetDelegationToken}} which has {{Joe}} as the owner > and {{oozie}} as the renewer in {{GetDelegationTokenReq}}. This API call > cannot instantiate a HMSClient and open transport to HMS using the HMSToken > string available in the sessionConf, since DBTokenStore uses server HiveConf > instead of sessionConf. It tries to establish transport using Kerberos and it > fails since user Joe is not Kerberos enabled. > I see the following exception trace in HS2 logs. > {noformat} > 2017-08-21T18:07:19,644 ERROR [HiveServer2-Handler-Pool: Thread-61] > transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) > ~[?:1.8.0_121] > at > org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at java.security.AccessController.doPrivileged(Native Method) > ~[?:1.8.0_121] >
[jira] [Commented] (HIVE-17368) DBTokenStore fails to connect in Kerberos enabled remote HMS environment
[ https://issues.apache.org/jira/browse/HIVE-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137758#comment-16137758 ] Hive QA commented on HIVE-17368: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12883234/HIVE-17368.01-branch-2.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10589 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explaindenpendencydiffengs] (batchId=38) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] (batchId=142) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic] (batchId=139) org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver (batchId=98) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs] (batchId=115) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_ptf] (batchId=125) org.apache.hadoop.hive.ql.security.TestExtendedAcls.testPartition (batchId=228) org.apache.hadoop.hive.ql.security.TestFolderPermissions.testPartition (batchId=217) org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=176) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6494/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6494/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6494/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 10 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12883234 - PreCommit-HIVE-Build > DBTokenStore fails to connect in Kerberos enabled remote HMS environment > > > Key: HIVE-17368 > URL: https://issues.apache.org/jira/browse/HIVE-17368 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0, 2.1.0, 2.2.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Attachments: HIVE-17368.01-branch-2.patch, HIVE-17368.01.patch, > HIVE-17368-branch-2.01.patch > > > In setups where HMS is running as a remote process secured using Kerberos, > and when {{DBTokenStore}} is configured as the token store, the HS2 Thrift > API calls like {{GetDelegationToken}}, {{CancelDelegationToken}} and > {{RenewDelegationToken}} fail with exception trace seen below. HS2 is not > able to invoke HMS APIs needed to add/remove/renew tokens from the DB since > it is possible that the user which is issue the {{GetDelegationToken}} is not > kerberos enabled. > Eg. Oozie submits a job on behalf of user "Joe". When Oozie opens a session > with HS2 it uses Oozie's principal and creates a proxy UGI with Hive. This > principal can establish a transport authenticated using Kerberos. It stores > the HMS delegation token string in the sessionConf and sessionToken. Now, > lets say Oozie issues a {{GetDelegationToken}} which has {{Joe}} as the owner > and {{oozie}} as the renewer in {{GetDelegationTokenReq}}. This API call > cannot instantiate a HMSClient and open transport to HMS using the HMSToken > string available in the sessionConf, since DBTokenStore uses server HiveConf > instead of sessionConf. It tries to establish transport using Kerberos and it > fails since user Joe is not Kerberos enabled. > I see the following exception trace in HS2 logs. > {noformat} > 2017-08-21T18:07:19,644 ERROR [HiveServer2-Handler-Pool: Thread-61] > transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) > ~[?:1.8.0_121] > at > org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) >
[jira] [Commented] (HIVE-17368) DBTokenStore fails to connect in Kerberos enabled remote HMS environment
[ https://issues.apache.org/jira/browse/HIVE-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136670#comment-16136670 ] Hive QA commented on HIVE-17368: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12883033/HIVE-17368.01.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10987 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning] (batchId=169) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=235) org.apache.hadoop.hive.common.TestFileUtils.testCopyWithDistCpAs (batchId=250) org.apache.hadoop.hive.common.TestFileUtils.testCopyWithDistcp (batchId=250) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=180) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=180) org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=180) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.org.apache.hive.minikdc.TestJdbcWithDBTokenStore (batchId=241) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6482/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6482/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6482/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 8 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12883033 - PreCommit-HIVE-Build > DBTokenStore fails to connect in Kerberos enabled remote HMS environment > > > Key: HIVE-17368 > URL: https://issues.apache.org/jira/browse/HIVE-17368 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.0.0, 2.1.0, 2.2.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Attachments: HIVE-17368.01.patch > > > In setups where HMS is running as a remote process secured using Kerberos, > and when {{DBTokenStore}} is configured as the token store, the HS2 Thrift > API calls like {{GetDelegationToken}}, {{CancelDelegationToken}} and > {{RenewDelegationToken}} fail with exception trace seen below. HS2 is not > able to invoke HMS APIs needed to add/remove/renew tokens from the DB since > it is possible that the user which is issue the {{GetDelegationToken}} is not > kerberos enabled. > Eg. Oozie submits a job on behalf of user "Joe". When Oozie opens a session > with HS2 it uses Oozie's principal and creates a proxy UGI with Hive. This > principal can establish a transport authenticated using Kerberos. It stores > the HMS delegation token string in the sessionConf and sessionToken. Now, > lets say Oozie issues a {{GetDelegationToken}} which has {{Joe}} as the owner > and {{oozie}} as the renewer in {{GetDelegationTokenReq}}. This API call > cannot instantiate a HMSClient and open transport to HMS using the HMSToken > string available in the sessionConf, since DBTokenStore uses server HiveConf > instead of sessionConf. It tries to establish transport using Kerberos and it > fails since user Joe is not Kerberos enabled. > I see the following exception trace in HS2 logs. > {noformat} > 2017-08-21T18:07:19,644 ERROR [HiveServer2-Handler-Pool: Thread-61] > transport.TSaslTransport: SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) > ~[?:1.8.0_121] > at > org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > [libthrift-0.9.3.jar:0.9.3] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) > [hive-shims-common-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at java.security.AccessController.doPrivileged(Native Method) > ~[?:1.8.0_121] > at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_121] > at >