Hi Kiran, can you enable SASL logging? Do it with "-Dsun.security.krb5.debug=true".
Alex On Fri, May 27, 2016 at 8:15 PM, kiran kumar <kiran.cse...@gmail.com> wrote: > Hi Alex, > > Thanks for the response. > > Here is the latest log. looks like it is failing at session establishment > after connection establishment successful. I have enabled sasl debug and > attached the log. > > [2016-05-27 23:03:39,280] INFO TGT valid starting at: Fri May 27 > 23:03:39 EDT 2016 (org.apache.zookeeper.Login) > [2016-05-27 23:03:39,280] INFO Opening socket connection to server > soa-test.mhf.mhc/10.201.186.73:4181. Will attempt to SASL-authenticate > using Login Context section 'Client' (org.apache.zookeeper.ClientCnxn) > [2016-05-27 23:03:39,280] INFO TGT expires: Sat May 28 > 23:03:39 EDT 2016 (org.apache.zookeeper.Login) > [2016-05-27 23:03:39,281] INFO TGT refresh sleeping until: Sat May 28 > 18:31:09 EDT 2016 (org.apache.zookeeper.Login) > [2016-05-27 23:03:39,285] INFO Socket connection established to > soa-test.mhf.mhc/10.201.186.73:4181, initiating session > (org.apache.zookeeper.ClientCnxn) > [2016-05-27 23:03:39,287] DEBUG Session establishment request sent on > soa-test.mhf.mhc/10.201.186.73:4181 (org.apache.zookeeper.ClientCnxn) > [2016-05-27 23:03:49,287] INFO Client session timed out, have not heard > from server in 10002ms for sessionid 0x0, closing socket connection and > attempting reconnect (org.apache.zookeeper.ClientCnxn) > [2016-05-27 23:03:49,999] INFO Client will use GSSAPI as SASL mechanism. > (org.apache.zookeeper.client.ZooKeeperSaslClient) > [2016-05-27 23:03:49,999] DEBUG creating sasl client: > client=kafka/r...@mhf.mhc;service=kafka;serviceHostname=soa-test.mhf.mhc > (org.apache.zookeeper.client.ZooKeeperSaslClient) > [2016-05-27 23:03:50,000] INFO Opening socket connection to server > soa-test.mhf.mhc/10.201.186.72:4181. Will attempt to SASL-authenticate > using Login Context section 'Client' (org.apache.zookeeper.ClientCnxn) > [2016-05-27 23:03:50,001] *INFO Socket connection established to > soa-test.mhf.mhc/10.201.186.72:4181 <http://10.201.186.72:4181>, initiating > session (org.apache.zookeeper.ClientCnxn)* > [2016-05-27 23:03:50,001] DEBUG Session establishment request sent on > soa-test.mhf.mhc/10.201.186.72:4181 (org.apache.zookeeper.ClientCnxn) > [2016-05-27 23:04:00,002]* INFO Client session timed out, have not heard > from server in 10001ms for sessionid 0x0, closing socket connection and > attempting reconnect (org.apache.zookeeper.ClientCnxn)* > [2016-05-27 23:04:00,186] INFO Client will use GSSAPI as SASL mechanism. > (org.apache.zookeeper.client.ZooKeeperSaslClient) > [2016-05-27 23:04:00,186] DEBUG creating sasl client: > client=kafka/r...@mhf.mhc;service=kafka;serviceHostname=nj09mhf5527 > (org.apache.zookeeper.client.ZooKeeperSaslClient) > [2016-05-27 23:04:00,187] INFO Opening socket connection to server > nj09mhf5527/10.201.186.71:4181. Will attempt to SASL-authenticate using > Login Context section 'Client' (org.apache.zookeeper.ClientCnxn) > [2016-05-27 23:04:00,188] *INFO Socket connection established to > nj09mhf5527/10.201.186.71:4181 <http://10.201.186.71:4181>, initiating > session (org.apache.zookeeper.ClientCnxn)* > [2016-05-27 23:04:00,188] DEBUG Session establishment request sent on > nj09mhf5527/10.201.186.71:4181 (org.apache.zookeeper.ClientCnxn) > [2016-05-27 23:04:09,045] DEBUG Closing ZkClient... > (org.I0Itec.zkclient.ZkClient) > [2016-05-27 23:04:09,046] INFO Terminate ZkClient event thread. > (org.I0Itec.zkclient.ZkEventThread) > [2016-05-27 23:04:09,046] DEBUG Closing ZooKeeper connected to > nj09mhf5527.mhf.mhc:4181,nj09mhf5528.mhf.mhc:4181,nj09mhf5529.mhf.mhc:4181 > (org.I0Itec.zkclient.ZkConnection) > [2016-05-27 23:04:09,046] DEBUG Closing session: 0x0 > (org.apache.zookeeper.ZooKeeper) > [2016-05-27 23:04:09,046] DEBUG Closing client for session: 0x0 > (org.apache.zookeeper.ClientCnxn) > [2016-05-27 23:04:10,187] DEBUG An exception was thrown while closing send > thread for session 0x0 : Client session timed out, have not heard from > server in 10000ms for sessionid 0x0 (org.apache.zookeeper.ClientCnxn) > [2016-05-27 23:04:10,293] DEBUG Disconnecting client for session: 0x0 > (org.apache.zookeeper.ClientCnxn) > [2016-05-27 23:04:10,293] INFO Session: 0x0 closed > (org.apache.zookeeper.ZooKeeper) > [2016-05-27 23:04:10,293] DEBUG Closing ZkClient...done > (org.I0Itec.zkclient.ZkClient) > [2016-05-27 23:04:10,294] INFO EventThread shut down > (org.apache.zookeeper.ClientCnxn) > [2016-05-27 23:04:10,295] FATAL Fatal error during KafkaServer startup. > Prepare to shutdown (kafka.server.KafkaServer) > org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to > zookeeper server within timeout: 30000 > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1232) > at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:156) > at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:130) > > > Thanks, > Kiran > > On Sat, May 28, 2016 at 4:15 AM, Alex Loddengaard <a...@confluent.io> > wrote: > >> Hi Kiran, >> >> Can you attach your configuration files not in a .zip? >> >> Most likely your broker isn't using the correct hostname:port to connect >> to >> ZooKeeper. Although if you're using ZooKeeper SASL, you may have a SASL >> misconfiguration. Set the `sun.security.krb5.debug` JVM property to `true` >> to get SASL debug messages. >> >> Alex >> >> On Fri, May 27, 2016 at 5:38 AM, kiran kumar <kiran.cse...@gmail.com> >> wrote: >> >> > Hello All, >> > >> > After enabling the security for the kafka brokers and zookeeper, I am >> > unable to start the kafka brokers. >> > >> > I have attached all the required configuration files. >> > >> > Please let me know if you need more information. >> > >> > Below is the complete error: >> > >> > FATAL Fatal error during KafkaServer startup. Prepare to shutdown >> > (kafka.server.KafkaServer) >> > org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to >> > zookeeper server within timeout: 30000 >> > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1232) >> > at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:156) >> > at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:130) >> > at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:75) >> > at kafka.utils.ZkUtils$.apply(ZkUtils.scala:57) >> > at kafka.server.KafkaServer.initZk(KafkaServer.scala:294) >> > at kafka.server.KafkaServer.startup(KafkaServer.scala:180) >> > at >> kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37) >> > at kafka.Kafka$.main(Kafka.scala:67) >> > at kafka.Kafka.main(Kafka.scala) >> > [2016-05-27 07:55:36,450] INFO shutting down (kafka.server.KafkaServer) >> > [2016-05-27 07:55:36,453] DEBUG Shutting down task scheduler. >> > (kafka.utils.KafkaScheduler) >> > [2016-05-27 07:55:36,456] INFO shut down completed >> > (kafka.server.KafkaServer) >> > [2016-05-27 07:55:36,458] FATAL Fatal error during KafkaServerStartable >> > startup. Prepare to shutdown (kafka.server.KafkaServerStartable) >> > org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to >> > zookeeper server within timeout: 30000 >> > at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1232) >> > at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:156) >> > at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:130) >> > at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:75) >> > at kafka.utils.ZkUtils$.apply(ZkUtils.scala:57) >> > at kafka.server.KafkaServer.initZk(KafkaServer.scala:294) >> > at kafka.server.KafkaServer.startup(KafkaServer.scala:180) >> > at >> kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37) >> > at kafka.Kafka$.main(Kafka.scala:67) >> > at kafka.Kafka.main(Kafka.scala) >> > >> > -- >> > G.Kiran Kumar >> > >> > > > > -- > G.Kiran Kumar >
