Re: New-Consumer group not showing up

2016-03-24 Thread craig w
Did you subscribe and poll? I believe your consumer group won't show up until it has been assigned one or more partitions. On Mar 24, 2016 9:48 PM, "Ryan Phillips" wrote: > I am only assigning this consumer one partition to listen to. Perhaps > that is where the issue lies, because the kafka-cons

Re: New-Consumer group not showing up

2016-03-24 Thread Ryan Phillips
I am only assigning this consumer one partition to listen to. Perhaps that is where the issue lies, because the kafka-console-consumer's group id (listening on all partitions) shows up correctly within the kafka-consumer-groups command. On Thu, Mar 24, 2016 at 6:59 PM, Ryan Phillips wrote: > Gree

Re: how to update Kafka WIKI with latest Ecosystem project info?

2016-03-24 Thread Marina
Hi, just wanted to bump it up again...I tried to see if I could edit the WIKI myself - but it seems that you need permissions to do so which I don't. How does one go about updating this part of the WIKI? thanks! From: Marina To: Users Sent: Tuesday, March 22, 2016 1:17 PM Subject: how

New-Consumer group not showing up

2016-03-24 Thread Ryan Phillips
Greetings, I’m attempting to use the New Consumer in my project (0.9.0.1), and it appears to be working nicely. The issue that I am seeing is that the group.id is not showing up within: kafka-consumer-groups.sh —bootstrap-server kafka0.local:9092,kafka1.local:9092 —new-consumer —list The command

Re: Messages corrupted in kafka

2016-03-24 Thread Becket Qin
You mentioned that you saw few corrupted messages, (< 0.1%). If so are you able to see some corrupted messages if you produce, say, 10M messages? On Wed, Mar 23, 2016 at 9:40 PM, sunil kalva wrote: > I am using java client and kafka 0.8.2, since events are corrupted in > kafka broker i cant rea

Re: consumer group, why commit requests are not considered as effective heartbeats?

2016-03-24 Thread Jason Gustafson
HI Zaiming, I think the problem is not that commit requests aren't considered as effective as heartbeats (they are), but that you can't rejoin the group using only commits/heartbeats. Every time the group rebalances, all members must rejoin the group by sending a JoinGroup request. Once a rebalanc

Reg : kafka consumer to dynamically detect topics added

2016-03-24 Thread Siddu Shebannavar
Hi Team, I'm using KafkaConsumer to consume messages from Kafka server (topics).. * It works fine for topics created before starting Consumer code... * But the problem is, it will not work if the topics created dynamically(i mean to say after consumer code started), but t

Reg. Kafka support on new RHEL

2016-03-24 Thread Shivegowda, Naveen (Nokia - IN/Bangalore)
Hi, I am planning to use Apache-Kafka on RHEL-7.x, so I wanted to know if it is supported on RHEL-7.x platform or it has any dependency on the 7.x OS ? Regards Naveen S

Re: Kafka 0.8.2.2 is using high CPU

2016-03-24 Thread li jinyu
add more information. we found that the linux(centos) running kafka has a very high context switch. and at first we thought it's caused by corrupted message data, but I tried to delete all data in one node and restarted kafka, but cpu usage is still high. following are outputs from sar and vmstat: