Hi Mikhail, thanks a lot for your help. One thing led to other and now we have the solution that I wanted to share with all.
We added the following in the code: System.setProperty("java.security.auth.login.config", "src/main/resources/hbase-jaas.conf"); System.setProperty("java.security.krb5.conf", "src/main/resources/krb5.conf"); And then we added those files in the src/main/resources. Everything else was the same and now our Java app can get the Kerberos ticket to proceed and connect. Best Regards, Jiten Sent from my iPhone > On Feb 11, 2015, at 10:09 PM, Mikhail Antonov <olorinb...@gmail.com> wrote: > > I'd say you don't need to have HBase cluster up and running at all to > be able to obtain kerberos ticket from standalone java app. > > On thing I noticed, which I overlooked before.. > > This piece of config containing hbase Configuration properties like > hbase.master.kerberos.principal etc shouldn't be needed in your custom > java app, right? All you need is a call to UGI.loginFromKeytab with > right principal and keytab file? > >> On Wed, Feb 11, 2015 at 9:38 PM, Jiten Gore <ji...@gores.net> wrote: >> The JAAS files on HBase Master, Region servers and Zookeeper do not >> currently exist. We will have to wait until tomorrow for their creation and >> further testing. >> >> Simply having the HBase-client.jaas on HBase client did not help. The error >> remains the same. >> >> Sent from my iPhone >> >>> On Feb 11, 2015, at 9:30 PM, Mikhail Antonov <olorinb...@gmail.com> wrote: >>> >>> Does error remain the same after changes in jaas config? >>> >>>> On Wed, Feb 11, 2015 at 7:56 PM, Jiten Gore <ji...@gores.net> wrote: >>>> The keytabs have been working for us when we use HBase shell as well as >>>> when we run pig scripts. >>>> >>>> Although our Java program is still unable to connect. >>>> >>>> Sent from my iPhone >>>> >>>>> On Feb 11, 2015, at 7:47 PM, Mikhail Antonov <olorinb...@gmail.com> wrote: >>>>> >>>>> I don't have any secured cluster handy to check and don't remember. I >>>>> supposed if you master and regionservers are starting fine and able to >>>>> login from keytabs than you're fine, otherwise you'll need to >>>>> configure jaas files for them. >>>>> >>>>> So does it work for you now? For your java program? >>>>> >>>>> -Mikhail >>>>> >>>>>> On Wed, Feb 11, 2015 at 7:40 PM, Jiten Gore <ji...@gores.net> wrote: >>>>>> This looks promising! >>>>>> >>>>>> On the host machine at /etc/hbase/conf, we have a jaas.conf file. >>>>>> >>>>>> It had useKeyTab = false >>>>>> We have changed it to: >>>>>> Client { >>>>>> com.sun.security.auth.module.Krb5LoginModule required >>>>>> useKeyTab=true >>>>>> keyTab=/home/<username>/username.keytab >>>>>> useTicketCache=true; >>>>>> }; >>>>>> >>>>>> Do we also need to add the other jaas files as shown here? >>>>>> https://ambari.apache.org/1.2.5/installing-hadoop-using-ambari/content/ambari-kerb-2-3-2-2.html >>>>>> >>>>>> >>>>>> >>>>>> Sent from my iPhone >>>>>> >>>>>>> On Feb 11, 2015, at 7:05 PM, Mikhail Antonov <olorinb...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>> at >>>>>>> com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:856) >>>>>>> at >>>>>>> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:719) >>>>>>> >>>>>>> Krb5LoginModule falls back to asking user for password when it's >>>>>>> either not configured to use keytabs, or can't find/read one. Do you >>>>>>> have JAAS conf file setup? You'd need to set useKeyTab=true and >>>>>>> keyTab=<path> there. >>>>>>> >>>>>>> -Mikhail >>>>>>> >>>>>>>> On Wed, Feb 11, 2015 at 6:50 PM, Jiten Gore <ji...@gores.net> wrote: >>>>>>>> Currently, running from a windows computer from within Eclipse. So >>>>>>>> permissions should not be an issue. >>>>>>>> >>>>>>>> Just set the property: >>>>>>>> System.setProperty("javax.security.auth.useSubjectCredsOnly", "false"); >>>>>>>> >>>>>>>> And got this output: >>>>>>>> Java config name: null >>>>>>>> Native config name: C:\Windows\krb5.ini >>>>>>>> getRealmFromDNS: trying <realm> >>>>>>>> getRealmFromDNS: trying <realm> >>>>>>>> Java config name: null >>>>>>>> Native config name: C:\Windows\krb5.ini >>>>>>>>>>> KdcAccessibility: reset >>>>>>>>>>> KdcAccessibility: reset >>>>>>>>>>> KeyTabInputStream, readName(): <REALM> >>>>>>>>>>> KeyTabInputStream, readName(): <username> >>>>>>>>>>> KeyTab: load() entry length: 53; type: 23 >>>>>>>>>>> KeyTabInputStream, readName(): <REALM> >>>>>>>>>>> KeyTabInputStream, readName(): <username> >>>>>>>>>>> KeyTab: load() entry length: 69; type: 18 >>>>>>>>>>> KeyTabInputStream, readName(): <REALM> >>>>>>>>>>> KeyTabInputStream, readName(): <username> >>>>>>>>>>> KeyTab: load() entry length: 53; type: 17 >>>>>>>> Ordering keys wrt default_tkt_enctypes list >>>>>>>> Using builtin default etypes for default_tkt_enctypes >>>>>>>> default etypes for default_tkt_enctypes: 17 16 23 1 3. >>>>>>>> Exception in thread "main" java.io.IOException: Login failure for >>>>>>>> <username>/<hostname>@<REALM> from keytab <path_to_keytab_file> >>>>>>>> at >>>>>>>> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1008) >>>>>>>> at Kerberos.KerberosAuthentication.App.hbase(App.java:44) >>>>>>>> at Kerberos.KerberosAuthentication.App.main(App.java:17) >>>>>>>> Caused by: javax.security.auth.login.LoginException: Unable to obtain >>>>>>>> password from user >>>>>>>> >>>>>>>> at >>>>>>>> com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:856) >>>>>>>> at >>>>>>>> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:719) >>>>>>>> at >>>>>>>> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:584) >>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>>>>> at >>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>>>>>>> at >>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606) >>>>>>>> at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762) >>>>>>>> at >>>>>>>> javax.security.auth.login.LoginContext.access$000(LoginContext.java:203) >>>>>>>> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690) >>>>>>>> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688) >>>>>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>>>>> at >>>>>>>> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687) >>>>>>>> at javax.security.auth.login.LoginContext.login(LoginContext.java:595) >>>>>>>> at >>>>>>>> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:997) >>>>>>>> ... 2 more >>>>>>>> LSA: Found Ticket >>>>>>>> LSA: Made NewWeakGlobalRef >>>>>>>> LSA: Found PrincipalName >>>>>>>> LSA: Made NewWeakGlobalRef >>>>>>>> LSA: Found DerValue >>>>>>>> LSA: Made NewWeakGlobalRef >>>>>>>> LSA: Found EncryptionKey >>>>>>>> LSA: Made NewWeakGlobalRef >>>>>>>> LSA: Found TicketFlags >>>>>>>> LSA: Made NewWeakGlobalRef >>>>>>>> LSA: Found KerberosTime >>>>>>>> LSA: Made NewWeakGlobalRef >>>>>>>> LSA: Found String >>>>>>>> LSA: Made NewWeakGlobalRef >>>>>>>> LSA: Found DerValue constructor >>>>>>>> LSA: Found Ticket constructor >>>>>>>> LSA: Found PrincipalName constructor >>>>>>>> LSA: Found EncryptionKey constructor >>>>>>>> LSA: Found TicketFlags constructor >>>>>>>> LSA: Found KerberosTime constructor >>>>>>>> LSA: Finished OnLoad processing >>>>>>>> >>>>>>>> >>>>>>>> Sent from my iPhone >>>>>>>> >>>>>>>>> On Feb 11, 2015, at 6:29 PM, Mikhail Antonov <olorinb...@gmail.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Interesting. >>>>>>>>> >>>>>>>>> Your java program runs under the same user, as shall for kinit? >>>>>>>>> Anything in /var/log/krb5kdc.log (with debug logging on)? >>>>>>>>> >>>>>>>>>> On Wed, Feb 11, 2015 at 6:17 PM, Jiten Gore <ji...@gores.net> wrote: >>>>>>>>>> The host names in libdefaults and realms in krb5.conf exactly match >>>>>>>>>> the host name used in the principal name. >>>>>>>>>> >>>>>>>>>> From command line, we are able to get the TGT using the following >>>>>>>>>> command: >>>>>>>>>> kinit -k -t <keytab> -p <username> >>>>>>>>>> >>>>>>>>>> Sent from my iPhone >>>>>>>>>> >>>>>>>>>>> On Feb 11, 2015, at 6:01 PM, Mikhail Antonov <olorinb...@gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Another thing to check are [libdefaults] and [realms] sections in >>>>>>>>>>> krb5.conf, in case there's any typo or wrong case in there. >>>>>>>>>>> >>>>>>>>>>> You can get the TGT from the kinit command using this keytab, right? >>>>>>>>>>> >>>>>>>>>>> -Mikhail >>>>>>>>>>> >>>>>>>>>>>> On Wed, Feb 11, 2015 at 5:58 PM, Mikhail Antonov >>>>>>>>>>>> <olorinb...@gmail.com> wrote: >>>>>>>>>>>> Just checking.. is that full log? Does the principal name have the >>>>>>>>>>>> _HOST portion in it? >>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Feb 11, 2015 at 5:24 PM, Jiten Gore <ji...@gores.net> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> Thanks Mikhail. Yes it has been so installed. >>>>>>>>>>>>> >>>>>>>>>>>>> We downloaded the JCE unlimited encryption jar files and replaced >>>>>>>>>>>>> the existing jre jar files. Is there any thing else that we need >>>>>>>>>>>>> to do? >>>>>>>>>>>>> >>>>>>>>>>>>> Sent from my iPhone >>>>>>>>>>>>> >>>>>>>>>>>>>> On Feb 11, 2015, at 5:08 PM, Mikhail Antonov >>>>>>>>>>>>>> <olorinb...@gmail.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Does your java app has JCE installed with unlimited encryption >>>>>>>>>>>>>> strength? >>>>>>>>>>>>>> >>>>>>>>>>>>>> -Mikhail >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Wed, Feb 11, 2015 at 4:52 PM, Jiten Gore <ji...@gores.net> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> Hi Dima, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks for the prompt response. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Here's what we are doing and the error we are seeing: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Code: >>>>>>>>>>>>>>> System.setProperty("javax.security.auth.useSubjectCredsOnly", >>>>>>>>>>>>>>> "false"); >>>>>>>>>>>>>>> final Configuration hBaseConfig = HBaseConfiguration.create(); >>>>>>>>>>>>>>> hBaseConfig.setInt("timeout", 120000); >>>>>>>>>>>>>>> hBaseConfig.set("hbase.zookeeper.quorum", "*************"); >>>>>>>>>>>>>>> hBaseConfig.set("hbase.zookeeper.property.clientPort", "2181"); >>>>>>>>>>>>>>> hBaseConfig.set("hadoop.security.authentication", "kerberos"); >>>>>>>>>>>>>>> hBaseConfig.set("hbase.security.authentication", "kerberos"); >>>>>>>>>>>>>>> hBaseConfig.set("hbase.master.kerberos.principal", >>>>>>>>>>>>>>> "*****************"); >>>>>>>>>>>>>>> hBaseConfig.set("hbase.regionserver.kerberos.principal", >>>>>>>>>>>>>>> "*******************"); >>>>>>>>>>>>>>> hBaseConfig.set("hbase.master.keytab.file", "hbase.keytab"); >>>>>>>>>>>>>>> hBaseConfig.set("hbase.regionserver.keytab.file", >>>>>>>>>>>>>>> "hbase.keytab"); >>>>>>>>>>>>>>> UserGroupInformation.setConfiguration(hBaseConfig); >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> UserGroupInformation ugi = >>>>>>>>>>>>>>> UserGroupInformation.loginUserFromKeytabAndReturnUGI("principle_name", >>>>>>>>>>>>>>> "user.keytab"); >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Error: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Exception in thread "main" java.io.IOException: Login failure >>>>>>>>>>>>>>> for <PRINCIPAL_NAME> from keytab >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1008) >>>>>>>>>>>>>>> at Kerberos.KerberosAuthentication.App.hbase(App.java:32) >>>>>>>>>>>>>>> at Kerberos.KerberosAuthentication.App.main(App.java:15) >>>>>>>>>>>>>>> Caused by: javax.security.auth.login.LoginException: null (68) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:763) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:584) >>>>>>>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>>>>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> javax.security.auth.login.LoginContext.invoke(LoginContext.java:762) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> javax.security.auth.login.LoginContext.access$000(LoginContext.java:203) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> javax.security.auth.login.LoginContext$4.run(LoginContext.java:690) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> javax.security.auth.login.LoginContext$4.run(LoginContext.java:688) >>>>>>>>>>>>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> javax.security.auth.login.LoginContext.login(LoginContext.java:595) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:997) >>>>>>>>>>>>>>> ... 2 more >>>>>>>>>>>>>>> Caused by: KrbException: null (68) >>>>>>>>>>>>>>> at sun.security.krb5.KrbAsRep.<init>(KrbAsRep.java:76) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> sun.security.krb5.KrbAsReqBuilder.send(KrbAsReqBuilder.java:319) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> sun.security.krb5.KrbAsReqBuilder.action(KrbAsReqBuilder.java:364) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:735) >>>>>>>>>>>>>>> ... 15 more >>>>>>>>>>>>>>> Caused by: KrbException: Identifier doesn't match expected >>>>>>>>>>>>>>> value (906) >>>>>>>>>>>>>>> at sun.security.krb5.internal.KDCRep.init(KDCRep.java:143) >>>>>>>>>>>>>>> at sun.security.krb5.internal.ASRep.init(ASRep.java:65) >>>>>>>>>>>>>>> at sun.security.krb5.internal.ASRep.<init>(ASRep.java:60) >>>>>>>>>>>>>>> at sun.security.krb5.KrbAsRep.<init>(KrbAsRep.java:60) >>>>>>>>>>>>>>> Sent from my iPhone >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Feb 11, 2015, at 10:56 AM, Dima Spivak >>>>>>>>>>>>>>>> <dspi...@cloudera.com> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hey Jiten, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Have you followed the steps outlined in >>>>>>>>>>>>>>>> http://hbase.apache.org/book.html#hbase.secure.configuration ? >>>>>>>>>>>>>>>> What issues >>>>>>>>>>>>>>>> are you seeing? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -Dima >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Wed, Feb 11, 2015 at 12:49 PM, Jiten Gore >>>>>>>>>>>>>>>>> <ji...@gores.net> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> We are having difficulties connecting with our Java >>>>>>>>>>>>>>>>> application to our >>>>>>>>>>>>>>>>> Kerberized HBase cluster. We are using a keytab file to >>>>>>>>>>>>>>>>> authenticate. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Has anyone successfully connected this way? If you have and >>>>>>>>>>>>>>>>> can help, >>>>>>>>>>>>>>>>> please let me know. I can share details about the issue. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Best Regards, >>>>>>>>>>>>>>>>> Jiten >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Sent from my iPhone >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Michael Antonov >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Michael Antonov >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Thanks, >>>>>>>>>>> Michael Antonov >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Thanks, >>>>>>>>> Michael Antonov >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Thanks, >>>>>>> Michael Antonov >>>>> >>>>> >>>>> >>>>> -- >>>>> Thanks, >>>>> Michael Antonov >>> >>> >>> >>> -- >>> Thanks, >>> Michael Antonov > > > > -- > Thanks, > Michael Antonov >