Well, I am running on the same machine, so I say yes Sent from my iPhone
> On Feb 23, 2016, at 18:05, Martin Gainty <mgai...@hotmail.com> wrote: > > one more thing to check: > > specifically are the /etc/krb5.conf credentials the same you use to > authenticate to ubuntu.oleg.com > > ? > Martin > __________________ > > > > >> Subject: Re: Kerberized Kafka setup issues >> From: ozhurakou...@hortonworks.com >> To: users@kafka.apache.org >> Date: Tue, 23 Feb 2016 21:58:48 +0000 >> >> Harsh >> >> I followed this blog >> (http://www.confluent.io/blog/apache-kafka-security-authorization-authentication-encryption) >> and got an environment via vagrant setup, no issues. I’ll poke around what >> the differences are and if find the issue will post. >> Thanks for your help anyway. >> >> Cheers >> Oleg >> On Feb 23, 2016, at 4:06 PM, Oleg Zhurakousky >> <ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote: >> >> Yeah, I noticed the localhost as well, but I’ve changed it since to FQDN and >> it is still the same including 'sname is >> zookeeper/localh...@oleg.com<mailto:zookeeper/localh...@oleg.com>’ >> >> Oleg >> >> On Feb 23, 2016, at 4:00 PM, Harsha >> <ka...@harsha.io<mailto:ka...@harsha.io>> wrote: >> >> whats your zookeeper.connect in server.properties looks like. Did you >> use the hostname or localhost >> -Harsha >> >> On Tue, Feb 23, 2016, at 12:01 PM, Oleg Zhurakousky wrote: >> Still digging, but here is more info that may help >> >> 2016-02-23 14:59:24,240] INFO zookeeper state changed (SyncConnected) >> (org.I0Itec.zkclient.ZkClient) >> Found ticket for >> kafka/ubuntu.oleg....@oleg.com<mailto:kafka/ubuntu.oleg....@oleg.com> to go >> to >> krbtgt/oleg....@oleg.com<mailto:krbtgt/oleg....@oleg.com> expiring on Wed >> Feb 24 00:59:24 EST 2016 >> Entered Krb5Context.initSecContext with state=STATE_NEW >> Found ticket for >> kafka/ubuntu.oleg....@oleg.com<mailto:kafka/ubuntu.oleg....@oleg.com> to go >> to >> krbtgt/oleg....@oleg.com<mailto:krbtgt/oleg....@oleg.com> expiring on Wed >> Feb 24 00:59:24 EST 2016 >> Service ticket not found in the subject >> Credentials acquireServiceCreds: same realm >> Using builtin default etypes for default_tgs_enctypes >> default etypes for default_tgs_enctypes: 17 16 23. >> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType >> EType: sun.security.krb5.internal.crypto.Des3CbcHmacSha1KdEType >> KrbKdcReq send: kdc=ubuntu.oleg.com<http://ubuntu.oleg.com> UDP:88, >> timeout=30000, number of retries =3, #bytes=660 >> KDCCommunication: kdc=ubuntu.oleg.com<http://ubuntu.oleg.com> UDP:88, >> timeout=30000,Attempt =1, #bytes=660 >> KrbKdcReq send: #bytes read=183 >> KdcAccessibility: remove ubuntu.oleg.com<http://ubuntu.oleg.com> >> KDCRep: init() encoding tag is 126 req type is 13 >> KRBError: >> cTime is Sat Aug 01 11:32:55 EDT 1998 901985575000 >> sTime is Tue Feb 23 14:59:24 EST 2016 1456257564000 >> suSec is 248635 >> error code is 7 >> error Message is Server not found in Kerberos database >> cname is >> kafka/ubuntu.oleg....@oleg.com<mailto:kafka/ubuntu.oleg....@oleg.com> >> sname is zookeeper/localh...@oleg.com<mailto:zookeeper/localh...@oleg.com> >> msgType is 30 >> >> On Feb 23, 2016, at 2:46 PM, Oleg Zhurakousky >> <ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote: >> >> No joy. the same error >> >> KafkaServer { >> com.sun.security.auth.module.Krb5LoginModule required >> debug=true >> useKeyTab=true >> storeKey=true >> keyTab="/home/oleg/kafka_2.10-0.9.0.1/config/security/kafka.keytab" >> >> principal="kafka/ubuntu.oleg....@oleg.com<mailto:kafka/ubuntu.oleg....@oleg.com>"; >> }; >> Client { >> com.sun.security.auth.module.Krb5LoginModule required >> debug=true >> useKeyTab=true >> serviceName=zookeeper >> storeKey=true >> keyTab="/home/oleg/kafka_2.10-0.9.0.1/config/security/kafka.keytab" >> >> principal="kafka/ubuntu.oleg....@oleg.com<mailto:kafka/ubuntu.oleg....@oleg.com>"; >> }; >> On Feb 23, 2016, at 2:41 PM, Harsha <m...@harsha.io<mailto:m...@harsha.io>> >> wrote: >> >> My bad it should be under Client section >> >> Client { >> com.sun.security.auth.module.Krb5LoginModule required >> debug=true >> useKeyTab=true >> storeKey=true >> serviceName=zookeeper >> keyTab="/home/oleg/kafka_2.10-0.9.0.1/config/security/kafka.keytab" >> >> principal="kafka/ubuntu.oleg....@oleg.com<mailto:kafka/ubuntu.oleg....@oleg.com>"; >> }; >> >> -Harsha >> >> On Tue, Feb 23, 2016, at 11:37 AM, Harsha wrote: >> can you try adding "serviceName=zookeeper" to KafkaServer section like >> KafkaServer { >> com.sun.security.auth.module.Krb5LoginModule required >> debug=true >> useKeyTab=true >> storeKey=true >> serviceName=zookeeper >> keyTab="/home/oleg/kafka_2.10-0.9.0.1/config/security/kafka.keytab" >> >> principal="kafka/ubuntu.oleg....@oleg.com<mailto:kafka/ubuntu.oleg....@oleg.com>"; >> }; >> >> On Tue, Feb 23, 2016, at 11:24 AM, Oleg Zhurakousky wrote: >> More info >> >> I am starting both services as myself ‘oleg’. Validated that both key tab >> files are readable. o I am assuming Zookeeper is started as ‘zookeeper’ >> and Kafka as ‘kafka’ >> >> Oleg >> >> On Feb 23, 2016, at 2:22 PM, Oleg Zhurakousky >> <ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote: >> >> Harsha >> >> Thanks for following up. Here is is: >> oleg@ubuntu:~/kafka_2.10-0.9.0.1/config$ cat kafka_server_jaas.conf >> KafkaServer { >> com.sun.security.auth.module.Krb5LoginModule required >> debug=true >> useKeyTab=true >> storeKey=true >> keyTab="/home/oleg/kafka_2.10-0.9.0.1/config/security/kafka.keytab" >> >> principal="kafka/ubuntu.oleg....@oleg.com<mailto:kafka/ubuntu.oleg....@oleg.com>"; >> }; >> Client { >> com.sun.security.auth.module.Krb5LoginModule required >> debug=true >> useKeyTab=true >> storeKey=true >> keyTab="/home/oleg/kafka_2.10-0.9.0.1/config/security/kafka.keytab" >> >> principal="kafka/ubuntu.oleg....@oleg.com<mailto:kafka/ubuntu.oleg....@oleg.com>"; >> }; >> >> oleg@ubuntu:~/kafka_2.10-0.9.0.1/config$ cat zookeeper_jaas.conf >> Server { >> com.sun.security.auth.module.Krb5LoginModule required >> debug=true >> useKeyTab=true >> keyTab="/home/oleg/kafka_2.10-0.9.0.1/config/security/zookeeper.keytab" >> storeKey=true >> useTicketCache=false >> principal="zookeeper/ubuntu.oleg....@oleg.com<mailto:zookeeper/ubuntu.oleg....@oleg.com>"; >> }; >> >> Cheers >> Oleg >> >> On Feb 23, 2016, at 2:17 PM, Harsha >> <ka...@harsha.io<mailto:ka...@harsha.io>> wrote: >> >> Oleg, >> Can you post your jaas configs. Its important that serviceName >> must match the principal name with which zookeeper is running. >> Whats the principal name zookeeper service is running with. >> -Harsha >> >> On Tue, Feb 23, 2016, at 11:01 AM, Oleg Zhurakousky wrote: >> Hey guys, first post here so bare with me >> >> Trying to setup Kerberized Kafka 0.9.0.. Followed the instructions here >> http://kafka.apache.org/documentation.html#security_sasl and i seem to be >> very close, but not quite there yet. >> >> ZOOKEEPER >> Starting Zookeeper seems to be OK (below is the relevant part of the log) >> . . . >> [2016-02-23 13:22:40,336] INFO maxSessionTimeout set to -1 >> (org.apache.zookeeper.server.ZooKeeperServer) >> Debug is true storeKey true useTicketCache false useKeyTab true >> doNotPrompt false ticketCache is null isInitiator true KeyTab is >> /home/oleg/kafka_2.10-0.9.0.1/config/security/zookeeper.keytab >> refreshKrb5Config is false principal is >> zookeeper/ubuntu.oleg....@oleg.com<mailto:zookeeper/ubuntu.oleg....@oleg.com><mailto:zookeeper/ubuntu.oleg....@oleg.com> >> tryFirstPass is false useFirstPass is false storePass is false clearPass >> is false >> principal is >> zookeeper/ubuntu.oleg....@oleg.com<mailto:zookeeper/ubuntu.oleg....@oleg.com><mailto:zookeeper/ubuntu.oleg....@oleg.com> >> Will use keytab >> Commit Succeeded >> >> [2016-02-23 13:22:40,541] INFO successfully logged in. >> (org.apache.zookeeper.Login) >> [2016-02-23 13:22:40,544] INFO binding to port 0.0.0.0/0.0.0.0:2181 >> (org.apache.zookeeper.server.NIOServerCnxnFactory) >> [2016-02-23 13:22:40,544] INFO TGT refresh thread started. >> (org.apache.zookeeper.Login) >> [2016-02-23 13:22:40,554] INFO TGT valid starting at: Tue Feb 23 >> 13:22:40 EST 2016 (org.apache.zookeeper.Login) >> [2016-02-23 13:22:40,554] INFO TGT expires: Tue Feb 23 >> 23:22:40 EST 2016 (org.apache.zookeeper.Login) >> [2016-02-23 13:22:40,554] INFO TGT refresh sleeping until: Tue Feb 23 >> 21:47:35 EST 2016 (org.apache.zookeeper.Login) >> [2016-02-23 13:23:09,012] INFO Accepted socket connection from >> /127.0.0.1:51876 (org.apache.zookeeper.server.NIOServerCnxnFactory) >> [2016-02-23 13:23:09,025] INFO Client attempting to establish new session >> at /127.0.0.1:51876 (org.apache.zookeeper.server.ZooKeeperServer) >> [2016-02-23 13:23:09,026] INFO Creating new log file: log.57 >> (org.apache.zookeeper.server.persistence.FileTxnLog) >> . . . >> >> >> KAFKA >> Starting Kafka server is not going well yet although I see that >> interaction with Kerberos is successful (see relevant log below. the >> error is at the bottom) >> . . . >> [2016-02-23 13:26:11,508] INFO starting (kafka.server.KafkaServer) >> [2016-02-23 13:26:11,511] INFO Connecting to zookeeper on localhost:2181 >> (kafka.server.KafkaServer) >> [2016-02-23 13:26:11,519] INFO JAAS File name: >> /home/oleg/kafka_2.10-0.9.0.1/config/kafka_server_jaas.conf >> (org.I0Itec.zkclient.ZkClient) >> [2016-02-23 13:26:11,520] INFO Starting ZkClient event thread. >> (org.I0Itec.zkclient.ZkEventThread) >> [2016-02-23 13:26:11,527] INFO Client >> environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 >> GMT (org.apache.zookeeper.ZooKeeper) >> [2016-02-23 13:26:11,527] INFO Client environment:host.name=172.16.137.20 >> (org.apache.zookeeper.ZooKeeper) >> [2016-02-23 13:26:11,527] INFO Client environment:java.version=1.8.0_72 >> (org.apache.zookeeper.ZooKeeper) >> [2016-02-23 13:26:11,527] INFO Client environment:java.vendor=Oracle >> Corporation (org.apache.zookeeper.ZooKeeper) >> [2016-02-23 13:26:11,527] INFO Client >> environment:java.home=/usr/lib/jvm/java-8-oracle/jre >> (org.apache.zookeeper.ZooKeeper) >> [2016-02-23 13:26:11,527] INFO Client >> environment:java.class.path=:/home/oleg/kafka_2.10-0.9.0.1/bin/../libs/jetty-http-9.2.12.v20150709.jar:/home/oleg/ka. >> . . . . . >> [2016-02-23 13:26:11,531] INFO Client >> environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib >> (org.apache.zookeeper.ZooKeeper) >> [2016-02-23 13:26:11,531] INFO Client environment:java.io.tmpdir=/tmp >> (org.apache.zookeeper.ZooKeeper) >> [2016-02-23 13:26:11,531] INFO Client environment:java.compiler=<NA> >> (org.apache.zookeeper.ZooKeeper) >> [2016-02-23 13:26:11,531] INFO Client environment:os.name=Linux >> (org.apache.zookeeper.ZooKeeper) >> [2016-02-23 13:26:11,531] INFO Client environment:os.arch=amd64 >> (org.apache.zookeeper.ZooKeeper) >> [2016-02-23 13:26:11,531] INFO Client >> environment:os.version=4.2.0-27-generic (org.apache.zookeeper.ZooKeeper) >> [2016-02-23 13:26:11,531] INFO Client environment:user.name=oleg >> (org.apache.zookeeper.ZooKeeper) >> [2016-02-23 13:26:11,531] INFO Client environment:user.home=/home/oleg >> (org.apache.zookeeper.ZooKeeper) >> [2016-02-23 13:26:11,531] INFO Client >> environment:user.dir=/home/oleg/kafka_2.10-0.9.0.1 >> (org.apache.zookeeper.ZooKeeper) >> [2016-02-23 13:26:11,532] INFO Initiating client connection, >> connectString=localhost:2181 sessionTimeout=6000 >> watcher=org.I0Itec.zkclient.ZkClient@647fd8ce >> (org.apache.zookeeper.ZooKeeper) >> [2016-02-23 13:26:11,541] INFO Waiting for keeper state SaslAuthenticated >> (org.I0Itec.zkclient.ZkClient) >> Debug is true storeKey true useTicketCache false useKeyTab true >> doNotPrompt false ticketCache is null isInitiator true KeyTab is >> /home/oleg/kafka_2.10-0.9.0.1/config/security/kafka.keytab >> refreshKrb5Config is false principal is >> kafka/ubuntu.oleg....@oleg.com<mailto:kafka/ubuntu.oleg....@oleg.com><mailto:kafka/ubuntu.oleg....@oleg.com> >> tryFirstPass is false useFirstPass is false storePass is false clearPass >> is false >> principal is >> kafka/ubuntu.oleg....@oleg.com<mailto:kafka/ubuntu.oleg....@oleg.com><mailto:kafka/ubuntu.oleg....@oleg.com> >> Will use keytab >> Commit Succeeded >> >> [2016-02-23 13:26:11,734] INFO successfully logged in. >> (org.apache.zookeeper.Login) >> [2016-02-23 13:26:11,735] INFO TGT refresh thread started. >> (org.apache.zookeeper.Login) >> [2016-02-23 13:26:11,738] INFO Client will use GSSAPI as SASL mechanism. >> (org.apache.zookeeper.client.ZooKeeperSaslClient) >> [2016-02-23 13:26:11,743] INFO Opening socket connection to server >> localhost/127.0.0.1:2181. Will attempt to SASL-authenticate using Login >> Context section 'Client' (org.apache.zookeeper.ClientCnxn) >> [2016-02-23 13:26:11,748] INFO Socket connection established to >> localhost/127.0.0.1:2181, initiating session >> (org.apache.zookeeper.ClientCnxn) >> [2016-02-23 13:26:11,752] INFO TGT valid starting at: Tue Feb 23 >> 13:26:11 EST 2016 (org.apache.zookeeper.Login) >> [2016-02-23 13:26:11,752] INFO TGT expires: Tue Feb 23 >> 23:26:11 EST 2016 (org.apache.zookeeper.Login) >> [2016-02-23 13:26:11,752] INFO TGT refresh sleeping until: Tue Feb 23 >> 21:40:22 EST 2016 (org.apache.zookeeper.Login) >> [2016-02-23 13:26:11,761] INFO Session establishment complete on server >> localhost/127.0.0.1:2181, sessionid = 0x1530f5e6fcb0001, negotiated >> timeout = 6000 (org.apache.zookeeper.ClientCnxn) >> [2016-02-23 13:26:11,762] INFO zookeeper state changed (SyncConnected) >> (org.I0Itec.zkclient.ZkClient) >> [2016-02-23 13:26:11,773] ERROR An error: >> (java.security.PrivilegedActionException: >> javax.security.sasl.SaslException: GSS initiate failed [Caused by >> GSSException: No valid credentials provided (Mechanism level: Server not >> found in Kerberos database (7) - LOOKING_UP_SERVER)]) occurred when >> evaluating Zookeeper Quorum Member's received SASL token. Zookeeper >> Client will go to AUTH_FAILED state. >> (org.apache.zookeeper.client.ZooKeeperSaslClient) >> [2016-02-23 13:26:11,773] ERROR SASL authentication with Zookeeper Quorum >> member failed: javax.security.sasl.SaslException: An error: >> (java.security.PrivilegedActionException: >> javax.security.sasl.SaslException: GSS initiate failed [Caused by >> GSSException: No valid credentials provided (Mechanism level: Server not >> found in Kerberos database (7) - LOOKING_UP_SERVER)]) occurred when >> evaluating Zookeeper Quorum Member's received SASL token. Zookeeper >> Client will go to AUTH_FAILED state. (org.apache.zookeeper.ClientCnxn) >> [2016-02-23 13:26:11,774] INFO zookeeper state changed (AuthFailed) >> (org.I0Itec.zkclient.ZkClient) >> [2016-02-23 13:26:17,542] INFO Terminate ZkClient event thread. >> (org.I0Itec.zkclient.ZkEventThread) >> . . . >> >> Any pointers? >> >> Cheers >> Oleg >
