[
https://issues.apache.org/jira/browse/KAFKA-3391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15245402#comment-15245402
]
Manikumar Reddy commented on KAFKA-3391:
----------------------------------------
increasing ZK timeout may help.
Kafka user mailing list can help you to solve this kind of issues.
https://kafka.apache.org/contact.html
> Kafka to ZK timeout
> --------------------
>
> Key: KAFKA-3391
> URL: https://issues.apache.org/jira/browse/KAFKA-3391
> Project: Kafka
> Issue Type: Bug
> Components: consumer, zkclient
> Affects Versions: 0.8.2.0
> Environment: RHEL 7.2, AWS EC2 compute instance
> Reporter: Karthik Reddy
> Assignee: Neha Narkhede
> Priority: Critical
>
> Hi Team,
> We have seen the below messages in the Kafka logs, indicating there was a
> timeout on ZK.
> Could you please advise us on how to tune or better optimize the Kafka-ZK
> communication.
> Kafka and ZK are on separate servers.Currently, we have the ZK timeout set to
> 6000 ms.
> Kafka servers have EBS volumes as the disk.
> We had to restart our consumers and ZK to resolve this issue.
> [2016-03-10 02:29:25,858] INFO Unable to read additional data from server
> sessionid 0x5531d0003f30030, likely server has closed socket, closing socket
> connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
> [2016-03-10 02:29:25,958] INFO zookeeper state changed (Disconnected)
> (org.I0Itec.zkclient.ZkClient)
> [2016-03-10 02:29:26,381] INFO Opening socket connection to server
> 10.200.77.74/10.200.77.74:8164. Will not attempt to authenticate using SASL
> (unknown error) (org.apache.zookeeper.ClientCnxn)
> [2016-03-10 02:29:26,382] INFO Socket connection established to
> 10.200.77.74/10.200.77.74:8164, initiating session
> (org.apache.zookeeper.ClientCnxn)
> [2016-03-10 02:29:26,385] INFO Session establishment complete on server
> 10.200.77.74/10.200.77.74:8164, sessionid = 0x5531d0003f30030, negotiated
> timeout = 6000 (org.apache.zookeeper.ClientCnxn)
> [2016-03-10 02:29:26,385] INFO zookeeper state changed (SyncConnected)
> (org.I0Itec.zkclient.ZkClient)
> [2016-03-10 02:29:30,961] INFO conflict in /controller data:
> {"version":1,"brokerid":3,"timestamp":"1457594970952"} stored data:
> {"version":1,"brokerid":5,"timestamp":"1457594970043"} (kafka.utils.ZkUtils$)
> [2016-03-10 02:29:30,969] INFO New leader is 5
> (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
> [2016-03-10 02:29:31,620] INFO [ReplicaFetcherManager on broker 3] Removed
> fetcher for partitions
> [__consumer_offsets,0],[fulfillment.payments.autopay.mongooperation.response,1],[__consumer_offsets,20],[__consumer_offsets,40]
> (kafka.server.ReplicaFetcherManager)
> [2016-03-10 02:29:31,621] INFO [ReplicaFetcherManager on broker 3] Removed
> fetcher for partitions
> [efit.framework.notification.error,1],[__consumer_offsets,15],[fulfillment.payments.autopay.processexception.notification,1],[__consumer_offsets,35]
> (kafka.server.ReplicaFetcherManager)
> [2016-03-10 02:29:31,621] INFO Truncating log
> efit.framework.notification.error-1 to offset 637. (kafka.log.Log)
> [2016-03-10 02:29:31,621] INFO Truncating log __consumer_offsets-15 to offset
> 0. (kafka.log.Log)
> [2016-03-10 02:29:31,622] INFO Truncating log
> fulfillment.payments.autopay.processexception.notification-1 to offset 0.
> (kafka.log.Log)
> [2016-03-10 02:29:31,622] INFO Truncating log __consumer_offsets-35 to offset
> 0. (kafka.log.Log)
> [2016-03-10 02:29:31,623] INFO Loading offsets from [__consumer_offsets,0]
> (kafka.server.OffsetManager)
> [2016-03-10 02:29:31,624] INFO Loading offsets from [__consumer_offsets,20]
> (kafka.server.OffsetManager)
> [2016-03-10 02:29:31,624] INFO Finished loading offsets from
> [__consumer_offsets,0] in 1 milliseconds. (kafka.server.OffsetManager)
> [2016-03-10 02:29:31,625] INFO Loading offsets from [__consumer_offsets,40]
> (kafka.server.OffsetManager)
> [2016-03-10 02:29:31,625] INFO Finished loading offsets from
> [__consumer_offsets,20] in 1 milliseconds. (kafka.server.OffsetManager)
> [2016-03-10 02:29:31,625] INFO Finished loading offsets from
> [__consumer_offsets,40] in 0 milliseconds. (kafka.server.OffsetManager)
> [2016-03-10 02:29:31,627] INFO [ReplicaFetcherManager on broker 3] Added
> fetcher for partitions List([[efit.framework.notification.error,1],
> initOffset 637 to broker id:1,host:10.200.77.78,port:8165] ,
> [[__consumer_offsets,15], initOffset 0 to broker
> id:1,host:10.200.77.78,port:8165] ,
> [[fulfillment.payments.autopay.processexception.notification,1], initOffset 0
> to broker id:5,host:10.200.75.150,port:8165] , [[__consumer_offsets,35],
> initOffset 0 to broker id:1,host:10.200.77.78,port:8165] )
> (kafka.server.ReplicaFetcherManager)
> [2016-03-10 02:29:31,627] INFO [ReplicaFetcherThread-0-2], Shutting down
> (kafka.server.ReplicaFetcherThread
> Thanks,
> Karthik
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
