Re: Need a help in understanding __consumer_offsets topic creation in Kafka Cluster

Hi, We are using Kafka 0.9.0.2, and by default log cleaner is not enabled. When we enable the log cleaner, internal topic “__consumer_offsets” (around 1TB in size)starts compaction, and during compaction we are unable to consume/produce messages. Also, consumer groups failed in leader election.

Re: Need a help in understanding __consumer_offsets topic creation in Kafka Cluster

We changed the policy to "delete" dynamically for the __consumer_offsets topic and it was a better option than doing a cluster restart after enabling log compaction. Also, we found problems when you are replicating to a log compacted topic from a non-compacted topic (which is leader). On Mon, Mar

Re: Need a help in understanding __consumer_offsets topic creation in Kafka Cluster

Thanks @Stevo Slavić *Thanks, Kunal* *+91-9958189589* *Data Analyst* *First Paper Publication : **http://dl.acm.org/citation.cfm?id=2790798 * *Blog:- **http://learnhardwithkunalgupta.blogspot.in * On

Re: Need a help in understanding __consumer_offsets topic creation in Kafka Cluster

You are affected by this 0.9.0.0 bug https://issues.apache.org/jira/browse/KAFKA-2988 It was fixed for 0.9.0.1. You could just apply same fix to your 0.9.0.0 cluster but I'd recommend upgrading to 0.9.0.1. Kind regards, Stevo Slavic. On Mon, Mar 14, 2016, 07:10 Kunal Gupta

Need a help in understanding __consumer_offsets topic creation in Kafka Cluster

Hi everyone, I am new here, recently join the group. I faced a problem in Kafka Cluster, a problem is described below. I am using Kafka version 0.9.0.0 We have established a Kafka Cluster of 3 machines where 2 machines are utilized for Kafka broker and same 3 machines utilized for zookeeper.