Mate, I tried your suggestions as well, - make sure you have "Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files" installed (I think you need them for AES256?) and your java security configs are OK -- the server and client are able to obtain the TGT individually which indicates that the issue is not with JCE. the issue is when the client tries to obtain a service ticket using the TGT it has already obtained. - run "klist -e -k /etc/krb5.keytab" to see if what encryptions you have in the keytabs -- keys are available for both server and client, for AES256 and AES128 - check if you have full export support in JCE by "java KeyLengthDetector" -- do you maybe have a link to a guide for this? i was not able to find one through google. running this command as-is gives an error: "Error: Could not find or load main class KeyLengthDetector" - Maybe you can try with different encryption types in kerberos configs / during keytab generation. -- i changed encryption type to AES128. generating a TGT using kinit is working as expected, i.e., it returns an AES128 encrypted ticket. but zookeeper server and client are still requesting for AES256 encrypted tickets. i must have missed a config somewhere, i'll try to figure that out. - trying to use a different java version (latest JDK patches have some known kerberos backward-incompatibilities) -- i'll need to explore this further and try it
On Thu, 11 Jun 2020 at 16:58, Aparajita Singh <aparajita.1...@gmail.com> wrote: > Thanks Mate and Arpit, I'll check out your suggestions. > > Jorn, > > 1. Did you register the service principal correctly in your AD/KDC? > 1. yes, the client and principals are registered in KDC. using > kinit with keytab on the remote server was generating the TGT as > expected. > 2. If AD then did you make sure that the attribute for the user is > activated to enable AES256 Kerberos auth? > 1. i'm using KDC, AES256 was enabled by default. i changed the > kdc.conf file to use only AES128 but zookeeper is still using AES256. > i'm > looking to see if there is a config i have missed out somewhere during > zookeeper startup which is forcing it to use AES256 always. > 3. Do you have unlimited crypto policies installed with your JDK? > 1. yes, i verified this by checking if US_export_policy.jar and > local_policy.jar files are present in > $JAVA_HOME/jre/lib/security/unlimited > 4. Is the keytab accessible to zk? > 1. yes > 5. Did you create keytab with AES256 encryption type? > 1. yes, keytab had 2 entries for each principal corresponding to > AES256 and AES128 > > > On Thu, 11 Jun 2020 at 16:30, Jörn Franke <jornfra...@gmail.com> wrote: > >> Kerberos can be quite a beast for any application. I managed to use >> Kerberos authentication for Zookeeper a couple of times. Usually the error >> messages in Java are meaningless. >> >> Did you register the service principal correctly in your AD/KDC? >> >> If AD then did you make sure that the attribute for the user is activated >> to enable AES256 Kerberos auth? >> >> Do you have unlimited crypto policies installed with your JDK? >> >> Is the keytab accessible to zk? >> >> Did you create keytab with AES256 encryption type? >> >> >> > Am 10.06.2020 um 14:35 schrieb Aparajita Singh < >> aparajita.1...@gmail.com>: >> > >> > >> >> >> >> >> >> Hi, >> >> >> >> I am trying to migrate an unauthenticated zookeeper cluster to a >> kerberos >> >> authenticated one. For the time being SSL is disabled. I have >> configured >> >> the server and client as described below but when SASL is enabled I am >> >> unable to retreive data using zookeeper shell client from the zookeeper >> >> server. Could I get some help in understanding why this is failing? >> >> >> >> server.log snippet >> >> >> >> 2020-06-10 17:09:01,263 - INFO [NIOServerCxn.Factory: >> >> 0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket >> >> connection from /127.0.0.1:44994 >> >> >> >> 2020-06-10 17:09:01,264 - INFO [NIOServerCxn.Factory: >> >> 0.0.0.0/0.0.0.0:2181:NIOServerCnxn@827] - Processing mntr command >> from / >> >> 127.0.0.1:44994 >> >> >> >> 2020-06-10 17:09:01,265 - INFO [Thread-5:NIOServerCnxn@1007] - Closed >> >> socket connection for client /127.0.0.1:44994 (no session established >> for >> >> client) >> >> >> >> 2020-06-10 17:09:26,647 - INFO [main:Environment@100] - Client >> >> environment:zookeeper.version=3.4.6-169--1, built on 02/10/2016 05:49 >> GMT >> >> >> >> 2020-06-10 17:09:26,649 - INFO [main:Environment@100] - Client >> >> environment:host.name=stage-kdc-zk-ivy >> >> >> >> 2020-06-10 17:09:26,649 - INFO [main:Environment@100] - Client >> >> environment:java.version=1.8.0_172 >> >> >> >> 2020-06-10 17:09:26,651 - INFO [main:Environment@100] - Client >> >> environment:java.vendor=Oracle Corporation >> >> >> >> 2020-06-10 17:09:26,651 - INFO [main:Environment@100] - Client >> >> environment:java.home=/usr/lib/jvm/oracle-java8-jdk-amd64/jre >> >> >> >> 2020-06-10 17:09:26,651 - INFO [main:Environment@100] - Client >> >> >> environment:java.class.path=/usr/hdp/2.4.0.0-169/zookeeper/bin/../build/classes:/usr/hdp/2.4.0.0-169/zookeeper/bin/../build/lib/*.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/xercesMinimal-1.9.6.2.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/wagon-provider-api-2.4.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/wagon-http-shared4-2.4.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/wagon-http-shared-1.0-beta-6.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/wagon-http-lightweight-1.0-beta-6.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/wagon-http-2.4.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/wagon-file-1.0-beta-6.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/plexus-utils-3.0.8.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/plexus-interpolation-1.11.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/plexus-container-default-1.0-alpha-9-stable-1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/usr/hdp/ >> 2.4.0. >> 0-169/zookeeper/bin/../lib/nekohtml-1.9.6.2.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/maven-settings-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/maven-repository-metadata-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/maven-project-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/maven-profile-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/maven-plugin-registry-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/maven-model-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/maven-error-diagnostics-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/maven-artifact-manager-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/maven-artifact-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/maven-ant-tasks-2.1.3.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/log4j-1.2.16.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/jsoup-1.7.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/httpcore-4.2.3.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/httpclient-4.2.3.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/commons-logging-1.1.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/commons-io-2.2.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/commons-codec-1.6.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/classworlds-1.1-alpha-2.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/backport-util-concurrent-3.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/apache-log4j-extras-1.2.17.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/ant-launcher-1.8.0.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/ant-1.8.0.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../zookeeper-3.4.6.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../src/java/lib/*.jar:/usr/hdp/2.4.0.0-169/zookeeper/conf::/usr/hdp/2.4.0.0-169/zookeeper/conf:/usr/hdp/2.4.0.0-169/zookeeper/zookeeper.jar:/usr/hdp/ >> 2.4.0. >> 0-169/zookeeper/zookeeper-3.4.6.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/slf4j-log4j12-1.6.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/slf4j-api-1.6.1.jar:/usr/hdp/ >> 2.4.0. >> 0-169/zookeeper/lib/classworlds-1.1-alpha-2.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-model-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/httpcore-4.2.3.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/plexus-container-default-1.0-alpha-9-stable-1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/ant-launcher-1.8.0.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/plexus-utils-3.0.8.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/wagon-http-2.4.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-settings-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/log4j-1.2.16.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/netty-3.7.0.Final.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/commons-codec-1.6.jar:/usr/hdp/ >> 2.4.0. >> 0-169/zookeeper/lib/commons-io-2.2.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/nekohtml-1.9.6.2.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/backport-util-concurrent-3.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/apache-log4j-extras-1.2.17.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/ant-1.8.0.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/xercesMinimal-1.9.6.2.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/commons-logging-1.1.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/httpclient-4.2.3.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-profile-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-error-diagnostics-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-project-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/jsoup-1.7.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/plexus-interpolation-1.11.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-plugin-registry-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/wagon-http-shared-1.0-beta-6.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-repository-metadata-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/wagon-http-lightweight-1.0-beta-6.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-ant-tasks-2.1.3.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/wagon-http-shared4-2.4.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/wagon-provider-api-2.4.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-artifact-manager-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-artifact-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/wagon-file-1.0-beta-6.jar:/usr/share/zookeeper/* >> >> >> >> 2020-06-10 17:09:26,651 - INFO [main:Environment@100] - Client >> >> >> environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib >> >> >> >> 2020-06-10 17:09:26,651 - INFO [main:Environment@100] - Client >> >> environment:java.io.tmpdir=/tmp >> >> >> >> 2020-06-10 17:09:26,651 - INFO [main:Environment@100] - Client >> >> environment:java.compiler=<NA> >> >> >> >> 2020-06-10 17:09:26,651 - INFO [main:Environment@100] - Client >> >> environment:os.name=Linux >> >> >> >> 2020-06-10 17:09:26,652 - INFO [main:Environment@100] - Client >> >> environment:os.arch=amd64 >> >> >> >> 2020-06-10 17:09:26,652 - INFO [main:Environment@100] - Client >> >> environment:os.version=4.9.0-9-amd64 >> >> >> >> 2020-06-10 17:09:26,652 - INFO [main:Environment@100] - Client >> >> environment:user.name=root >> >> >> >> 2020-06-10 17:09:26,652 - INFO [main:Environment@100] - Client >> >> environment:user.home=/root >> >> >> >> 2020-06-10 17:09:26,652 - INFO [main:Environment@100] - Client >> >> environment:user.dir=/home/aparajita.singh >> >> >> >> 2020-06-10 17:09:26,653 - INFO [main:ZooKeeper@438] - Initiating >> client >> >> connection, connectString=stage-kdc-zk-ivy sessionTimeout=30000 >> >> watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@379619aa >> >> >> >> 2020-06-10 17:09:26,752 - INFO >> >> [main-SendThread(stage-kdc-zk-ivy:2181):Login@293] - successfully >> logged >> >> in. >> >> >> >> 2020-06-10 17:09:26,753 - INFO [Thread-0:Login$1@127] - TGT refresh >> >> thread started. >> >> >> >> 2020-06-10 17:09:26,757 - INFO >> >> [main-SendThread(stage-kdc-zk-ivy:2181):ZooKeeperSaslClient$1@285] - >> >> Client will use GSSAPI as SASL mechanism. >> >> >> >> 2020-06-10 17:09:26,758 - INFO [Thread-0:Login@301] - TGT valid >> starting >> >> at: Wed Jun 10 15:17:21 IST 2020 >> >> >> >> 2020-06-10 17:09:26,758 - INFO [Thread-0:Login@302] - TGT expires: >> >> Thu Jun 11 15:17:21 IST 2020 >> >> >> >> 2020-06-10 17:09:26,758 - INFO [Thread-0:Login$1@181] - TGT refresh >> >> sleeping until: Thu Jun 11 11:17:04 IST 2020 >> >> >> >> 2020-06-10 17:09:26,799 - INFO >> >> [main-SendThread(stage-kdc-zk-ivy:2181):ClientCnxn$SendThread@1019] - >> >> Opening socket connection to server stage-kdc-zk-ivy/ >> 10.33.203.225:2181. >> >> Will attempt to SASL-authenticate using Login Context section 'Client' >> >> >> >> 2020-06-10 17:09:26,854 - INFO [NIOServerCxn.Factory: >> >> 0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket >> >> connection from /10.33.203.225:45018 >> >> >> >> 2020-06-10 17:09:26,854 - INFO >> >> [main-SendThread(stage-kdc-zk-ivy:2181):ClientCnxn$SendThread@864] - >> >> Socket connection established to stage-kdc-zk-ivy/10.33.203.225:2181, >> >> initiating session >> >> >> >> 2020-06-10 17:09:26,856 - INFO [NIOServerCxn.Factory: >> >> 0.0.0.0/0.0.0.0:2181:ZooKeeperServer@868] - Client attempting to >> >> establish new session at /10.33.203.225:45018 >> >> >> >> 2020-06-10 17:09:26,859 - INFO [CommitProcessor:88:ZooKeeperServer@617 >> ] >> >> - Established session 0x58729e0540980002 with negotiated timeout 30000 >> for >> >> client /10.33.203.225:45018 >> >> >> >> 2020-06-10 17:09:26,861 - INFO >> >> [main-SendThread(stage-kdc-zk-ivy:2181):ClientCnxn$SendThread@1279] - >> >> Session establishment complete on server stage-kdc-zk-ivy/ >> >> 10.33.203.225:2181, sessionid = 0x58729e0540980002, negotiated >> timeout = >> >> 30000 >> >> >> >> 2020-06-10 17:09:27,007 - WARN [NIOServerCxn.Factory: >> >> 0.0.0.0/0.0.0.0:2181:ZooKeeperServer@969] - Client failed to SASL >> >> authenticate: javax.security.sasl.SaslException: GSS initiate failed >> >> [Caused by GSSException: Failure unspecified at GSS-API level >> (Mechanism >> >> level: Invalid argument (400) - Cannot find key of appropriate type to >> >> decrypt AP REP - AES256 CTS mode with HMAC SHA1-96)] >> >> >> >> 2020-06-10 17:09:27,007 - WARN [NIOServerCxn.Factory: >> >> 0.0.0.0/0.0.0.0:2181:ZooKeeperServer@975] - Closing client connection >> due >> >> to SASL authentication failure. >> >> >> >> 2020-06-10 17:09:27,007 - INFO [NIOServerCxn.Factory: >> >> 0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1007] - Closed socket connection >> for >> >> client /10.33.203.225:45018 which had sessionid 0x58729e0540980002 >> >> >> >> 2020-06-10 17:09:27,008 - ERROR [NIOServerCxn.Factory: >> >> 0.0.0.0/0.0.0.0:2181:NIOServerCnxn@178] - Unexpected Exception: >> >> >> >> java.nio.channels.CancelledKeyException >> >> >> >> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73) >> >> >> >> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77) >> >> >> >> at >> >> >> org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151) >> >> >> >> at >> >> >> org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081) >> >> >> >> at >> >> >> org.apache.zookeeper.server.ZooKeeperServer.processPacket(ZooKeeperServer.java:936) >> >> >> >> at >> >> >> org.apache.zookeeper.server.NIOServerCnxn.readRequest(NIOServerCnxn.java:373) >> >> >> >> at >> >> >> org.apache.zookeeper.server.NIOServerCnxn.readPayload(NIOServerCnxn.java:200) >> >> >> >> at >> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:244) >> >> >> >> at >> >> >> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) >> >> >> >> at java.lang.Thread.run(Thread.java:748) >> >> >> >> 2020-06-10 17:09:27,008 - INFO >> >> [main-SendThread(stage-kdc-zk-ivy:2181):ClientCnxn$SendThread@1142] - >> >> Unable to read additional data from server sessionid >> 0x58729e0540980002, >> >> likely server has closed socket, closing socket connection and >> attempting >> >> reconnect >> >> >> >> 2020-06-10 17:09:27,008 - WARN [NIOServerCxn.Factory: >> >> 0.0.0.0/0.0.0.0:2181:NIOServerCnxn@346] - Exception causing close of >> >> session 0x58729e0540980002 due to >> java.nio.channels.CancelledKeyException >> >> >> >> 2020-06-10 17:10:01,317 - INFO [NIOServerCxn.Factory: >> >> 0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket >> >> connection from /127.0.0.1:45004 >> >> >> >> 2020-06-10 17:10:01,318 - INFO [NIOServerCxn.Factory: >> >> 0.0.0.0/0.0.0.0:2181:NIOServerCnxn@827] - Processing mntr command >> from / >> >> 127.0.0.1:45004 >> >> >> >> >> >> >> >> zookeeper shell client output >> >> >> >> aparajita.singh@stage-kdc-zk-ivy:~$ sudo >> >> /usr/hdp/2.4.0.0-169/zookeeper/bin/zookeeper-client -server >> >> stage-kdc-zk-ivy get /test2 >> >> >> >> log4j:WARN Large window sizes are not allowed. >> >> >> >> log4j:WARN MaxIndex reduced to 13. >> >> >> >> Connecting to stage-kdc-zk-ivy >> >> >> >> Debug is true storeKey false useTicketCache true useKeyTab true >> >> doNotPrompt true ticketCache is /tmp/krb5cc_0 isInitiator true KeyTab >> is >> >> /etc/krb5.keytab refreshKrb5Config is false principal is >> >> zookeeper/stage-kdc-zk-...@stage.fdp.kafka tryFirstPass is false >> >> useFirstPass is false storePass is false clearPass is false >> >> >> >> Acquire TGT from Cache >> >> >> >> Principal is zookeeper/stage-kdc-zk-...@stage.fdp.kafka >> >> >> >> null credentials from Ticket Cache >> >> >> >> principal is zookeeper/stage-kdc-zk-...@stage.fdp.kafka >> >> >> >> Will use keytab >> >> >> >> Commit Succeeded >> >> >> >> >> >> >> >> WATCHER:: >> >> >> >> >> >> WatchedEvent state:SyncConnected type:None path:null >> >> >> >> >> >> WATCHER:: >> >> >> >> >> >> WatchedEvent state:Disconnected type:None path:null >> >> >> >> Exception in thread "main" >> >> org.apache.zookeeper.KeeperException$ConnectionLossException: >> >> KeeperErrorCode = ConnectionLoss for /test2 >> >> >> >> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) >> >> >> >> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) >> >> >> >> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155) >> >> >> >> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1184) >> >> >> >> at >> org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:717) >> >> >> >> at >> org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:591) >> >> >> >> at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:354) >> >> >> >> at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:282) >> >> >> >> zoo.cfg >> >> >> >> #setACL=False >> >> >> >> autopurge.snapRetainCount=30 >> >> >> >> tickTime=2000 >> >> >> >> dataDir=/grid/1/var/lib/zookeeper >> >> >> >> zookeeper_jmx_port=9009 >> >> >> >> initLimit=100 >> >> >> >> syncLimit=5 >> >> >> >> autopurge.purgeInterval=24 >> >> >> >> clientPort=2181 >> >> >> >> globalOutstandingLimit=5000 >> >> >> >> maxClientCnxns=2000 >> >> >> >> server.99=stage-kdc-zk-harley:2888:3888 >> >> >> >> server.88=stage-kdc-zk-ivy:2888:3888 >> >> >> >> server.77=stage-kdc-zk-2face:2888:3888 >> >> >> >> >> >> >> authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider >> >> >> >> requireClientAuthScheme=sasl >> >> >> >> >> >> quorum.auth.enableSasl=true >> >> >> >> quorum.auth.learnerRequireSasl=true >> >> >> >> quorum.auth.serverRequireSasl=true >> >> >> >> >> quorum.auth.kerberos.servicePrincipal=host/stage-kdc-zk-...@stage.fdp.kafka >> >> >> >> quorum.cnxn.threads.size=20 >> >> >> >> >> >> >> >> java.env >> >> >> >> SERVER_JVMFLAGS="${SERVER_JVMFLAGS} >> >> -Djava.security.auth.login.config=/home/aparajita.singh/jaas/jaas.conf >> >> >> -Dzookeeper.authProvider.sasl=org.apache.zookeeper.server.auth.SASLAuthenticationProvider >> >> -Dsun.security.krb5.debug=true" >> >> >> >> CLIENT_JVMFLAGS="${CLIENT_JVMFLAGS} >> >> >> -Djava.security.auth.login.config=/home/aparajita.singh/jaas/client.conf >> >> >> -Dzookeeper.authProvider.sasl=org.apache.zookeeper.server.auth.SASLAuthenticationProvider >> >> -Dsun.security.krb5.debug=true" >> >> >> >> >> >> /home/aparajita.singh/jaas/jaas.conf >> >> >> >> // Zookeeper server authentication >> >> >> >> Server { >> >> >> >> com.sun.security.auth.module.Krb5LoginModule required >> >> >> >> useKeyTab=true >> >> >> >> useTicketCache=false >> >> >> >> //ticketCache="/tmp/krb5cc_0" >> >> >> >> renewTicket=true >> >> >> >> doNotPrompt=true >> >> >> >> debug=true >> >> >> >> keyTab="/etc/krb5.keytab" >> >> >> >> serviceName="host" >> >> >> >> principal="host/stage-kdc-zk-...@stage.fdp.kafka"; >> >> >> >> }; >> >> >> >> >> >> // Zookeeper quorum server authentication >> >> >> >> QuorumServer { >> >> >> >> com.sun.security.auth.module.Krb5LoginModule required >> >> >> >> useKeyTab=true >> >> >> >> useTicketCache=false >> >> >> >> //ticketCache="/tmp/krb5cc_0" >> >> >> >> renewTicket=true >> >> >> >> doNotPrompt=true >> >> >> >> debug=true >> >> >> >> keyTab="/etc/krb5.keytab" >> >> >> >> serviceName="host" >> >> >> >> principal="host/stage-kdc-zk-...@stage.fdp.kafka"; >> >> >> >> }; >> >> >> >> >> >> // Zookeeper learner authentication >> >> >> >> QuorumLearner { >> >> >> >> com.sun.security.auth.module.Krb5LoginModule required >> >> >> >> useKeyTab=true >> >> >> >> useTicketCache=false >> >> >> >> //ticketCache="/tmp/krb5cc_0" >> >> >> >> renewTicket=true >> >> >> >> doNotPrompt=true >> >> >> >> debug=true >> >> >> >> keyTab="/etc/krb5.keytab" >> >> >> >> serviceName="host" >> >> >> >> principal="host/stage-kdc-zk-...@stage.fdp.kafka"; >> >> >> >> }; >> >> >> >> >> >> >> >> /home/aparajita.singh/jaas/client.conf >> >> >> >> // Zookeeper client authentication >> >> >> >> Client { >> >> >> >> com.sun.security.auth.module.Krb5LoginModule required >> >> >> >> useKeyTab=true >> >> >> >> useTicketCache=true >> >> >> >> ticketCache="/tmp/krb5cc_0" >> >> >> >> renewTicket=true >> >> >> >> doNotPrompt=true >> >> >> >> debug=true >> >> >> >> keyTab="/etc/krb5.keytab" >> >> >> >> serviceName="zookeeper" >> >> >> >> principal="zookeeper/stage-kdc-zk-...@stage.fdp.kafka"; >> >> >> >> }; >> >> >> >> >> >> Using kinit command I am able to generate the TGT for both principals. >> As >> >> per the zookeeper server log, the TGT can be generated as expected. The >> >> keytab file is accessible to all system users for now. >> >> >> >> aparajita.singh@stage-kdc-zk-ivy:~$ sudo /krb5/bin/kinit >> >> zookeeper/stage-kdc-zk-...@stage.fdp.kafka -k -t /etc/krb5.keytab >> >> >> >> aparajita.singh@stage-kdc-zk-ivy:~$ sudo /krb5/bin/kinit >> >> host/stage-kdc-zk-...@stage.fdp.kafka -k -t /etc/krb5.keytab >> >> >> >> >> >> -- >> >> Thanks, >> >> Aparajita >> >> >> > > > -- > Thanks, > Aparajita > -- Thanks, Aparajita