> 1) Hive CLI does not talk to HiveServer2 Oh yes. Absolutely. Sorry typo on my end.
> 2) Beeline talks to HiveServer2 and needs some way to authenticate itself depending on the configuration of HS2. HS2 can be configured to authenticate in one of these ways if I'm up to date: * NOSASL: no password needed * KERBEROS (SASL): no password needed So this is what confused me so I started digging deeper to understand what's going on and how it is exactly using the kerberos credentials for getting the connection. So it turns out that it actually boils down to using the HiveConnection[1] and depending on the auth type, opens up a transport to the HS. So technically it's not attempting to read credentials off the kerb ticket for the connection but completely ignoring the creds that are passed by the user, ofcourse unless it's SASL. That said, +1 to adding a check that we are using kerberos and skipping the prompt if we are. I think we probably don't even need to parse the URL to detect that. Just checking on the auth type property( hive.server2.authentication) is KERBEROS or not should do the trick. [1] https://github.com/apache/hive/blob/3991dba30c5068cac296f32e24e97cf87efa266c/jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java#L450-L455 On Wed, Aug 26, 2015 at 5:40 PM, Lars Francke <lars.fran...@gmail.com> wrote: > > On Wed, Aug 26, 2015 at 4:53 PM, kulkarni.swar...@gmail.com < > kulkarni.swar...@gmail.com> wrote: > >> > my understanding is that after using kerberos authentication, you >> probably don’t need the password. >> >> That is not an accurate statement. Beeline is a JDBC client as compared >> to Hive CLI which is a thrift client to talk to HIveServer2. So it would >> need the password to establish that JDBC connection. If you look at the >> beeline console code[1], it actually first tries to read the >> "javax.jdo.option.ConnectionUserName" and >> "javax.jdo.option.ConnectionPassword" property which is the same username >> and password that you have setup your backing metastore DB with. If it is >> MySWL, it would be the password you set MySQL with or empty if you >> haven't(or are using derby). Kerberos is merely a tool for you to >> authenticate yourself so that you cannot impersonate yourself as someone >> else. >> > > I don't think what you're saying is accurate. > > 1) Hive CLI does not talk to HiveServer2 > > 2) Beeline talks to HiveServer2 and needs some way to authenticate itself > depending on the configuration of HS2. > > HS2 can be configured to authenticate in one of these ways if I'm up to > date: > > * NOSASL: no password needed > * KERBEROS (SASL): no password needed > * NONE (SASL) using the AnonymousAuthenticationProviderImpl: no password > needed > * LDAP (SASL) using the LdapAuthenticationProviderImpl: username and > password required > * PAM (SASL) using the PamAuthenticationProviderImpl: username and > password required > * CUSTOM (SASL) using the CustomAuthenticationProviderImpl: username and > password required > > By tar the most common configurations are NONE (default I think) and > KERBEROS. Both don't need a username and password provided so it does not > make sense to ask for one every time. > > The only good reason I can think of to ask for a password is so that it > doesn't appear in a shell/beeline history and/or on screen. I'm sure there > are others? > The username can be safely provided in the URL if needed so I don't think > asking for that every time is reasonable either. > > What would be a good way to deal with this? I'm tempted to just rip out > those prompts. The other option would be to parse the connection URL and > check whether it's the Kerberos mode. > >> >> [1] >> https://github.com/apache/hive/blob/3991dba30c5068cac296f32e24e97cf87efa266c/beeline/src/java/org/apache/hive/beeline/Commands.java#L1117-L1125 >> >> On Wed, Aug 26, 2015 at 10:13 AM, Loïc Chanel < >> loic.cha...@telecomnancy.net> wrote: >> >>> Here it is : https://issues.apache.org/jira/browse/HIVE-11653 >>> >>> Loïc CHANEL >>> Engineering student at TELECOM Nancy >>> Trainee at Worldline - Villeurbanne >>> >>> 2015-08-25 23:10 GMT+02:00 Sergey Shelukhin <ser...@hortonworks.com>: >>> >>>> Sure! >>>> >>>> From: Loïc Chanel <loic.cha...@telecomnancy.net> >>>> Reply-To: "user@hive.apache.org" <user@hive.apache.org> >>>> Date: Tuesday, August 25, 2015 at 00:23 >>>> >>>> To: "user@hive.apache.org" <user@hive.apache.org> >>>> Subject: Re: HiveServer2 & Kerberos >>>> >>>> It is the case. >>>> Would you like me to fill a JIRA about it ? >>>> >>>> Loïc CHANEL >>>> Engineering student at TELECOM Nancy >>>> Trainee at Worldline - Villeurbanne >>>> >>>> 2015-08-24 19:24 GMT+02:00 Sergey Shelukhin <ser...@hortonworks.com>: >>>> >>>>> If that is the case it sounds like a bug… >>>>> >>>>> From: Jary Du <jary...@gmail.com> >>>>> Reply-To: "user@hive.apache.org" <user@hive.apache.org> >>>>> Date: Thursday, August 20, 2015 at 08:56 >>>>> To: "user@hive.apache.org" <user@hive.apache.org> >>>>> Subject: Re: HiveServer2 & Kerberos >>>>> >>>>> My understanding is that it will always ask you user/password even >>>>> though you don’t need them. It is just the way how hive is setup. >>>>> >>>>> On Aug 20, 2015, at 8:28 AM, Loïc Chanel <loic.cha...@telecomnancy.net> >>>>> wrote: >>>>> >>>>> !connect jdbc:hive2:// >>>>> 192.168.6.210:10000/db;principal=hive/hiveh...@westeros.wl >>>>> org.apache.hive.jdbc.HiveDriver >>>>> scan complete in 13ms >>>>> Connecting to jdbc:hive2:// >>>>> 192.168.6.210:10000/db;principal=hive/hiveh...@westeros.wl >>>>> Enter password for jdbc:hive2:// >>>>> 192.168.6.210:10000/chaneldb;principal=hive/hiveh...@westeros.wl: >>>>> >>>>> And if I press enter everything works perfectly, because I am using >>>>> Kerberos authentication, that's actually why I was asking what is Hive >>>>> asking for, because in my case, it seems that I shouldn't be asked for a >>>>> password when connecting. >>>>> >>>>> Loïc CHANEL >>>>> Engineering student at TELECOM Nancy >>>>> Trainee at Worldline - Villeurbanne >>>>> >>>>> 2015-08-20 17:06 GMT+02:00 Jary Du <jary...@gmail.com>: >>>>> >>>>>> How does Beeline ask you? What happens if you just press enter? >>>>>> >>>>>> >>>>>> >>>>>> On Aug 20, 2015, at 12:15 AM, Loïc Chanel < >>>>>> loic.cha...@telecomnancy.net> wrote: >>>>>> >>>>>> Indeed, I don't need the password, but why is Beeline asking me for >>>>>> one ? To what does it correspond ? >>>>>> >>>>>> Thanks again, >>>>>> >>>>>> >>>>>> Loïc >>>>>> >>>>>> Loïc CHANEL >>>>>> Engineering student at TELECOM Nancy >>>>>> Trainee at Worldline - Villeurbanne >>>>>> >>>>>> 2015-08-19 18:22 GMT+02:00 Jary Du <jary...@gmail.com>: >>>>>> >>>>>>> Correct me if I am wrong, my understanding is that after using >>>>>>> kerberos authentication, you probably don’t need the password. >>>>>>> >>>>>>> Hope it helps >>>>>>> >>>>>>> Thanks, >>>>>>> Jary >>>>>>> >>>>>>> >>>>>>> On Aug 19, 2015, at 9:09 AM, Loïc Chanel < >>>>>>> loic.cha...@telecomnancy.net> wrote: >>>>>>> >>>>>>> By the way, thanks a lot for your help, because your solution works, >>>>>>> but I'm still interested in knowing what is the password I did not >>>>>>> enter. >>>>>>> >>>>>>> Thanks again, >>>>>>> >>>>>>> >>>>>>> Loïc >>>>>>> >>>>>>> Loïc CHANEL >>>>>>> Engineering student at TELECOM Nancy >>>>>>> Trainee at Worldline - Villeurbanne >>>>>>> >>>>>>> 2015-08-19 18:07 GMT+02:00 Loïc Chanel <loic.cha...@telecomnancy.net >>>>>>> >: >>>>>>> >>>>>>>> All right, but then, what is the password hive asks for ? Hive's >>>>>>>> one ? How do I know its value ? >>>>>>>> >>>>>>>> Loïc CHANEL >>>>>>>> Engineering student at TELECOM Nancy >>>>>>>> Trainee at Worldline - Villeurbanne >>>>>>>> >>>>>>>> 2015-08-19 17:51 GMT+02:00 Jary Du <jary...@gmail.com>: >>>>>>>> >>>>>>>>> For Beeline connection string, it should be "!connect >>>>>>>>> jdbc:hive2://<host>:<port>/<db>;principal=<Server_Principal_of_HiveServer2>”. >>>>>>>>> Please >>>>>>>>> make sure it is the hive’s principal, not the user’s. And when you >>>>>>>>> kinit, >>>>>>>>> it should be kinit user’s keytab, not the hive’s keytab. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Aug 19, 2015, at 8:46 AM, Loïc Chanel < >>>>>>>>> loic.cha...@telecomnancy.net> wrote: >>>>>>>>> >>>>>>>>> Yeah, I forgot to mention it, but each time I did a kinit >>>>>>>>> user/hive before launching beeline, as I read somewhere that Beeline >>>>>>>>> does >>>>>>>>> not handle Kerberos connection. >>>>>>>>> >>>>>>>>> So, as I can make klist before launching beeline and having a good >>>>>>>>> result, the problem does not come from this. Thanks a lot for your >>>>>>>>> response >>>>>>>>> though. >>>>>>>>> Do you have another idea ? >>>>>>>>> >>>>>>>>> Loïc CHANEL >>>>>>>>> Engineering student at TELECOM Nancy >>>>>>>>> Trainee at Worldline - Villeurbanne >>>>>>>>> >>>>>>>>> 2015-08-19 17:42 GMT+02:00 Jary Du <jary...@gmail.com>: >>>>>>>>> >>>>>>>>>> "The Beeline client must have a valid Kerberos ticket in the >>>>>>>>>> ticket cache before attempting to connect." ( >>>>>>>>>> http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.3/bk_dataintegration/content/ch_using-hive-clients-examples.html >>>>>>>>>> ) >>>>>>>>>> >>>>>>>>>> So you need kinit first to have the valid Kerberos ticket int the >>>>>>>>>> ticket cache before using beeline to connect to HS2. >>>>>>>>>> >>>>>>>>>> Jary >>>>>>>>>> >>>>>>>>>> On Aug 19, 2015, at 8:36 AM, Loïc Chanel < >>>>>>>>>> loic.cha...@telecomnancy.net> wrote: >>>>>>>>>> >>>>>>>>>> Hi again, >>>>>>>>>> >>>>>>>>>> As I searched another way to make some requests with Kerberos >>>>>>>>>> enabled for security on HiveServer, I found that this request should >>>>>>>>>> do the >>>>>>>>>> same : >>>>>>>>>> !connect jdbc:hive2:// >>>>>>>>>> 192.168.6.210:10000/default;principal=user/h...@westeros.wl >>>>>>>>>> org.apache.hive.jdbc.HiveDriver >>>>>>>>>> But now I've got another error : >>>>>>>>>> Error: Could not open client transport with JDBC Uri: >>>>>>>>>> jdbc:hive2:// >>>>>>>>>> 192.168.6.210:10000/default;principal=user/h...@westeros.wl: >>>>>>>>>> Peer indicated failure: GSS initiate failed (state=08S01,code=0) >>>>>>>>>> >>>>>>>>>> As I saw that it was maybe a simple Kerberos ticket related >>>>>>>>>> problem, I tried to re-generate Kerberos keytabs, and to ensure that >>>>>>>>>> Hive >>>>>>>>>> has the path to access to its keytab, but nothing changed. >>>>>>>>>> >>>>>>>>>> Does anyone has an idea about how to solve this issue ? >>>>>>>>>> >>>>>>>>>> Thanks in advance for your help :) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Loïc >>>>>>>>>> >>>>>>>>>> Loïc CHANEL >>>>>>>>>> Engineering student at TELECOM Nancy >>>>>>>>>> Trainee at Worldline - Villeurbanne >>>>>>>>>> >>>>>>>>>> 2015-08-19 12:01 GMT+02:00 Loïc Chanel < >>>>>>>>>> loic.cha...@telecomnancy.net>: >>>>>>>>>> >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> I have a little issue with HiveServer2 since I have enabled >>>>>>>>>>> Kerberos. I'm unable to connect to the service via Beeline. When >>>>>>>>>>> doing >>>>>>>>>>> !connect jdbc:hive2://192.168.6.210:10000 hive hive >>>>>>>>>>> org.apache.hive.jdbc.HiveDriver >>>>>>>>>>> I keep receiving the same error : >>>>>>>>>>> Error: Could not open client transport with JDBC Uri: >>>>>>>>>>> jdbc:hive2://192.168.6.210:10000: Peer indicated failure: >>>>>>>>>>> Unsupported mechanism type PLAIN (state=08S01,code=0) >>>>>>>>>>> >>>>>>>>>>> Does anyone had the same problem ? Or know how to solve it ? >>>>>>>>>>> Thanks in advance, >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Loïc >>>>>>>>>>> >>>>>>>>>>> Loïc CHANEL >>>>>>>>>>> Engineering student at TELECOM Nancy >>>>>>>>>>> Trainee at Worldline - Villeurbanne >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >>> >> >> >> -- >> Swarnim >> > > -- Swarnim
