Re: One Kafka Broker Went Rogue

2016-12-29 Thread Tony Liu
​Hi, I just found a reported issue, https://issues.apache.org/jira/browse/KAFKA-4477, hopefully, it's useful for you.​ On Thu, Dec 29, 2016 at 12:08 PM, Tony Liu wrote: > Hi Thomas or Anyone, > > I also encountered the same issue like you reported, the only workaround > is to restart that broke

Re: One Kafka Broker Went Rogue

2016-12-29 Thread Tony Liu
Hi Thomas or Anyone, I also encountered the same issue like you reported, the only workaround is to restart that broken node, but I did not find root cause how to solve it right now, so I wonder do you have some progress how to solve that issue right now? i.e, at the beginning, I thought this iss

Re: One Kafka Broker Went Rogue

2016-12-06 Thread Thomas DeVoe
Hi All, This happened again on our kafka cluster - a single kafka broker seems to "forget" the existence of the rest of the cluster and shrinks all of its ISRs to only exist on that node. The other two nodes get stuck in a loop trying to connect to this rogue node and never even register that it i

One Kafka Broker Went Rogue

2016-11-29 Thread Thomas DeVoe
Hi, I encountered a strange issue in our kafka cluster, where randomly a single broker entered a state where it seemed to think it was the only broker in the cluster (it shrank all of its ISRs to just existing on itself). Some details about the kafka cluster: - running in an EC2 VPC on AWS - 3 no