Hi guys ! Sorry to interrupt but I need to go back to the first reason of this thread : I can't connect to hive anymore. I upgraded my cluster to HDP 2.3, and I saw that the way to connect to Hive via Beeline & Kerberos hasn't changed, but the exact command that worked before doesn't work anymore. Instead of connecting, Beeline returns me : Error: Failed to open new session: java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive/hiveserverh...@example.com is not allowed to impersonate testUser (state=,code=0)
The logs are not more explicit, as there is an exception with the same conclusion : Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive/hiveserverh...@example.com is not allowed to impersonate testUser Do any of you have an idea about where this could come from ? Loïc CHANEL Engineering student at TELECOM Nancy Trainee at Worldline - Villeurbanne 2015-08-31 13:51 GMT+02:00 Lars Francke <lars.fran...@gmail.com>: > That said, +1 to adding a check that we are using kerberos and skipping >> the prompt if we are. I think we probably don't even need to parse the URL >> to detect that. Just checking on the auth type property( >> hive.server2.authentication) is KERBEROS or not should do the trick. >> > > I have not looked into this at all but Beeline being a generic client does > it even use that property? I mean I could connect to any server, right? > Will try to take a look. > > >> [1] >> https://github.com/apache/hive/blob/3991dba30c5068cac296f32e24e97cf87efa266c/jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java#L450-L455 >> >> On Wed, Aug 26, 2015 at 5:40 PM, Lars Francke <lars.fran...@gmail.com> >> wrote: >> >>> >>> On Wed, Aug 26, 2015 at 4:53 PM, kulkarni.swar...@gmail.com < >>> kulkarni.swar...@gmail.com> wrote: >>> >>>> > my understanding is that after using kerberos authentication, you >>>> probably don’t need the password. >>>> >>>> That is not an accurate statement. Beeline is a JDBC client as compared >>>> to Hive CLI which is a thrift client to talk to HIveServer2. So it would >>>> need the password to establish that JDBC connection. If you look at the >>>> beeline console code[1], it actually first tries to read the >>>> "javax.jdo.option.ConnectionUserName" and >>>> "javax.jdo.option.ConnectionPassword" property which is the same username >>>> and password that you have setup your backing metastore DB with. If it is >>>> MySWL, it would be the password you set MySQL with or empty if you >>>> haven't(or are using derby). Kerberos is merely a tool for you to >>>> authenticate yourself so that you cannot impersonate yourself as someone >>>> else. >>>> >>> >>> I don't think what you're saying is accurate. >>> >>> 1) Hive CLI does not talk to HiveServer2 >>> >>> 2) Beeline talks to HiveServer2 and needs some way to authenticate >>> itself depending on the configuration of HS2. >>> >>> HS2 can be configured to authenticate in one of these ways if I'm up to >>> date: >>> >>> * NOSASL: no password needed >>> * KERBEROS (SASL): no password needed >>> * NONE (SASL) using the AnonymousAuthenticationProviderImpl: no >>> password needed >>> * LDAP (SASL) using the LdapAuthenticationProviderImpl: username and >>> password required >>> * PAM (SASL) using the PamAuthenticationProviderImpl: username and >>> password required >>> * CUSTOM (SASL) using the CustomAuthenticationProviderImpl: username >>> and password required >>> >>> By tar the most common configurations are NONE (default I think) and >>> KERBEROS. Both don't need a username and password provided so it does not >>> make sense to ask for one every time. >>> >>> The only good reason I can think of to ask for a password is so that it >>> doesn't appear in a shell/beeline history and/or on screen. I'm sure there >>> are others? >>> The username can be safely provided in the URL if needed so I don't >>> think asking for that every time is reasonable either. >>> >>> What would be a good way to deal with this? I'm tempted to just rip out >>> those prompts. The other option would be to parse the connection URL and >>> check whether it's the Kerberos mode. >>> >>>> >>>> [1] >>>> https://github.com/apache/hive/blob/3991dba30c5068cac296f32e24e97cf87efa266c/beeline/src/java/org/apache/hive/beeline/Commands.java#L1117-L1125 >>>> >>>> On Wed, Aug 26, 2015 at 10:13 AM, Loïc Chanel < >>>> loic.cha...@telecomnancy.net> wrote: >>>> >>>>> Here it is : https://issues.apache.org/jira/browse/HIVE-11653 >>>>> >>>>> Loïc CHANEL >>>>> Engineering student at TELECOM Nancy >>>>> Trainee at Worldline - Villeurbanne >>>>> >>>>> 2015-08-25 23:10 GMT+02:00 Sergey Shelukhin <ser...@hortonworks.com>: >>>>> >>>>>> Sure! >>>>>> >>>>>> From: Loïc Chanel <loic.cha...@telecomnancy.net> >>>>>> Reply-To: "user@hive.apache.org" <user@hive.apache.org> >>>>>> Date: Tuesday, August 25, 2015 at 00:23 >>>>>> >>>>>> To: "user@hive.apache.org" <user@hive.apache.org> >>>>>> Subject: Re: HiveServer2 & Kerberos >>>>>> >>>>>> It is the case. >>>>>> Would you like me to fill a JIRA about it ? >>>>>> >>>>>> Loïc CHANEL >>>>>> Engineering student at TELECOM Nancy >>>>>> Trainee at Worldline - Villeurbanne >>>>>> >>>>>> 2015-08-24 19:24 GMT+02:00 Sergey Shelukhin <ser...@hortonworks.com>: >>>>>> >>>>>>> If that is the case it sounds like a bug… >>>>>>> >>>>>>> From: Jary Du <jary...@gmail.com> >>>>>>> Reply-To: "user@hive.apache.org" <user@hive.apache.org> >>>>>>> Date: Thursday, August 20, 2015 at 08:56 >>>>>>> To: "user@hive.apache.org" <user@hive.apache.org> >>>>>>> Subject: Re: HiveServer2 & Kerberos >>>>>>> >>>>>>> My understanding is that it will always ask you user/password even >>>>>>> though you don’t need them. It is just the way how hive is setup. >>>>>>> >>>>>>> On Aug 20, 2015, at 8:28 AM, Loïc Chanel < >>>>>>> loic.cha...@telecomnancy.net> wrote: >>>>>>> >>>>>>> !connect jdbc:hive2:// >>>>>>> 192.168.6.210:10000/db;principal=hive/hiveh...@westeros.wl >>>>>>> org.apache.hive.jdbc.HiveDriver >>>>>>> scan complete in 13ms >>>>>>> Connecting to jdbc:hive2:// >>>>>>> 192.168.6.210:10000/db;principal=hive/hiveh...@westeros.wl >>>>>>> Enter password for jdbc:hive2:// >>>>>>> 192.168.6.210:10000/chaneldb;principal=hive/hiveh...@westeros.wl: >>>>>>> >>>>>>> And if I press enter everything works perfectly, because I am using >>>>>>> Kerberos authentication, that's actually why I was asking what is Hive >>>>>>> asking for, because in my case, it seems that I shouldn't be asked for a >>>>>>> password when connecting. >>>>>>> >>>>>>> Loïc CHANEL >>>>>>> Engineering student at TELECOM Nancy >>>>>>> Trainee at Worldline - Villeurbanne >>>>>>> >>>>>>> 2015-08-20 17:06 GMT+02:00 Jary Du <jary...@gmail.com>: >>>>>>> >>>>>>>> How does Beeline ask you? What happens if you just press enter? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Aug 20, 2015, at 12:15 AM, Loïc Chanel < >>>>>>>> loic.cha...@telecomnancy.net> wrote: >>>>>>>> >>>>>>>> Indeed, I don't need the password, but why is Beeline asking me for >>>>>>>> one ? To what does it correspond ? >>>>>>>> >>>>>>>> Thanks again, >>>>>>>> >>>>>>>> >>>>>>>> Loïc >>>>>>>> >>>>>>>> Loïc CHANEL >>>>>>>> Engineering student at TELECOM Nancy >>>>>>>> Trainee at Worldline - Villeurbanne >>>>>>>> >>>>>>>> 2015-08-19 18:22 GMT+02:00 Jary Du <jary...@gmail.com>: >>>>>>>> >>>>>>>>> Correct me if I am wrong, my understanding is that after using >>>>>>>>> kerberos authentication, you probably don’t need the password. >>>>>>>>> >>>>>>>>> Hope it helps >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Jary >>>>>>>>> >>>>>>>>> >>>>>>>>> On Aug 19, 2015, at 9:09 AM, Loïc Chanel < >>>>>>>>> loic.cha...@telecomnancy.net> wrote: >>>>>>>>> >>>>>>>>> By the way, thanks a lot for your help, because your solution >>>>>>>>> works, but I'm still interested in knowing what is the password I did >>>>>>>>> not >>>>>>>>> enter. >>>>>>>>> >>>>>>>>> Thanks again, >>>>>>>>> >>>>>>>>> >>>>>>>>> Loïc >>>>>>>>> >>>>>>>>> Loïc CHANEL >>>>>>>>> Engineering student at TELECOM Nancy >>>>>>>>> Trainee at Worldline - Villeurbanne >>>>>>>>> >>>>>>>>> 2015-08-19 18:07 GMT+02:00 Loïc Chanel < >>>>>>>>> loic.cha...@telecomnancy.net>: >>>>>>>>> >>>>>>>>>> All right, but then, what is the password hive asks for ? Hive's >>>>>>>>>> one ? How do I know its value ? >>>>>>>>>> >>>>>>>>>> Loïc CHANEL >>>>>>>>>> Engineering student at TELECOM Nancy >>>>>>>>>> Trainee at Worldline - Villeurbanne >>>>>>>>>> >>>>>>>>>> 2015-08-19 17:51 GMT+02:00 Jary Du <jary...@gmail.com>: >>>>>>>>>> >>>>>>>>>>> For Beeline connection string, it should be "!connect >>>>>>>>>>> jdbc:hive2://<host>:<port>/<db>;principal=<Server_Principal_of_HiveServer2>”. >>>>>>>>>>> Please >>>>>>>>>>> make sure it is the hive’s principal, not the user’s. And when you >>>>>>>>>>> kinit, >>>>>>>>>>> it should be kinit user’s keytab, not the hive’s keytab. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Aug 19, 2015, at 8:46 AM, Loïc Chanel < >>>>>>>>>>> loic.cha...@telecomnancy.net> wrote: >>>>>>>>>>> >>>>>>>>>>> Yeah, I forgot to mention it, but each time I did a kinit >>>>>>>>>>> user/hive before launching beeline, as I read somewhere that >>>>>>>>>>> Beeline does >>>>>>>>>>> not handle Kerberos connection. >>>>>>>>>>> >>>>>>>>>>> So, as I can make klist before launching beeline and having a >>>>>>>>>>> good result, the problem does not come from this. Thanks a lot for >>>>>>>>>>> your >>>>>>>>>>> response though. >>>>>>>>>>> Do you have another idea ? >>>>>>>>>>> >>>>>>>>>>> Loïc CHANEL >>>>>>>>>>> Engineering student at TELECOM Nancy >>>>>>>>>>> Trainee at Worldline - Villeurbanne >>>>>>>>>>> >>>>>>>>>>> 2015-08-19 17:42 GMT+02:00 Jary Du <jary...@gmail.com>: >>>>>>>>>>> >>>>>>>>>>>> "The Beeline client must have a valid Kerberos ticket in the >>>>>>>>>>>> ticket cache before attempting to connect." ( >>>>>>>>>>>> http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.3/bk_dataintegration/content/ch_using-hive-clients-examples.html >>>>>>>>>>>> ) >>>>>>>>>>>> >>>>>>>>>>>> So you need kinit first to have the valid Kerberos ticket int >>>>>>>>>>>> the ticket cache before using beeline to connect to HS2. >>>>>>>>>>>> >>>>>>>>>>>> Jary >>>>>>>>>>>> >>>>>>>>>>>> On Aug 19, 2015, at 8:36 AM, Loïc Chanel < >>>>>>>>>>>> loic.cha...@telecomnancy.net> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi again, >>>>>>>>>>>> >>>>>>>>>>>> As I searched another way to make some requests with Kerberos >>>>>>>>>>>> enabled for security on HiveServer, I found that this request >>>>>>>>>>>> should do the >>>>>>>>>>>> same : >>>>>>>>>>>> !connect jdbc:hive2:// >>>>>>>>>>>> 192.168.6.210:10000/default;principal=user/h...@westeros.wl >>>>>>>>>>>> org.apache.hive.jdbc.HiveDriver >>>>>>>>>>>> But now I've got another error : >>>>>>>>>>>> Error: Could not open client transport with JDBC Uri: >>>>>>>>>>>> jdbc:hive2:// >>>>>>>>>>>> 192.168.6.210:10000/default;principal=user/h...@westeros.wl: >>>>>>>>>>>> Peer indicated failure: GSS initiate failed (state=08S01,code=0) >>>>>>>>>>>> >>>>>>>>>>>> As I saw that it was maybe a simple Kerberos ticket related >>>>>>>>>>>> problem, I tried to re-generate Kerberos keytabs, and to ensure >>>>>>>>>>>> that Hive >>>>>>>>>>>> has the path to access to its keytab, but nothing changed. >>>>>>>>>>>> >>>>>>>>>>>> Does anyone has an idea about how to solve this issue ? >>>>>>>>>>>> >>>>>>>>>>>> Thanks in advance for your help :) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Loïc >>>>>>>>>>>> >>>>>>>>>>>> Loïc CHANEL >>>>>>>>>>>> Engineering student at TELECOM Nancy >>>>>>>>>>>> Trainee at Worldline - Villeurbanne >>>>>>>>>>>> >>>>>>>>>>>> 2015-08-19 12:01 GMT+02:00 Loïc Chanel < >>>>>>>>>>>> loic.cha...@telecomnancy.net>: >>>>>>>>>>>> >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> I have a little issue with HiveServer2 since I have enabled >>>>>>>>>>>>> Kerberos. I'm unable to connect to the service via Beeline. When >>>>>>>>>>>>> doing >>>>>>>>>>>>> !connect jdbc:hive2://192.168.6.210:10000 hive hive >>>>>>>>>>>>> org.apache.hive.jdbc.HiveDriver >>>>>>>>>>>>> I keep receiving the same error : >>>>>>>>>>>>> Error: Could not open client transport with JDBC Uri: >>>>>>>>>>>>> jdbc:hive2://192.168.6.210:10000: Peer indicated failure: >>>>>>>>>>>>> Unsupported mechanism type PLAIN (state=08S01,code=0) >>>>>>>>>>>>> >>>>>>>>>>>>> Does anyone had the same problem ? Or know how to solve it ? >>>>>>>>>>>>> Thanks in advance, >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Loïc >>>>>>>>>>>>> >>>>>>>>>>>>> Loïc CHANEL >>>>>>>>>>>>> Engineering student at TELECOM Nancy >>>>>>>>>>>>> Trainee at Worldline - Villeurbanne >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Swarnim >>>> >>> >>> >> >> >> -- >> Swarnim >> > >
