Re: open source projects based on kafka

Hi Li, You might take a look at Apache Samza. It's conceptually simple but powerful and makes heavy use of Kafka. Best, Roger Sent from my iPhone > On Sep 12, 2015, at 10:34 PM, Li Tao wrote: > > Hi Hackers, > > This is Lee, a learner of kafka, i have read the original paper on kafka, > a

open source projects based on kafka

Hi Hackers, This is Lee, a learner of kafka, i have read the original paper on kafka, and walked through the document. I think the best way to learn sth is to write and read code about it. I am wondering is there any open source code / system which is based on kafka so that i can read or contribu

Broker restart, new producer and snappy in 0.8.2.1

There has been multiple instances of this incident. When I restart the broker for config changes it's bringing down the consumers and mirror maker and I am seeing CRC corruption in mirror maker and following error in broker. I have reset the offset in zookeeper for certain topic/partitions. But I s

Re: Zookeeper jmx monitoring for kafka

Fyi regarding disk space utilization, I submitted a patch to track ZK log and snapshot directory sizes: ZOOKEEPER-1423 Best, Eddie Em 12/09/2015 11:44, "Flavio Junqueira" escreveu: > We do support jmx extensively, and I believe you can use it Ganglia. As > for metrics, things that come to mind n

Re: 0.9.0.0 remaining jiras

Excuse me, I'd put my issue in this list on the backburner, but resuming work on it this week. Cheers, Eddie Em 12/09/2015 13:40, "Jun Rao" escreveu: > The following is a candidate list of jiras that we want to complete in the > upcoming release (0.9.0.0). Our goal is to finish at least all the

0.9.0.0 remaining jiras

The following is a candidate list of jiras that we want to complete in the upcoming release (0.9.0.0). Our goal is to finish at least all the blockers and as many as the non-blockers possible in that list. https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20status%20in%20(Open

Re: Zookeeper jmx monitoring for kafka

We do support jmx extensively, and I believe you can use it Ganglia. As for metrics, things that come to mind now on top of what Otis suggested are: - Frequency of leader election (if they happen too frequently, then that's going to affect your availability and probably an indication of somethin

Re: automatically consume from all topics

It is handled by the implementation of the high level consumer automatically. High level consumer internal registers an listener on ZK topic znode. Any changes to it including topic creation high level consumer will be notify and and then rebalanced topics if the new topic matches the regex. You ca

Re: Possible DEAD LOCK for one day at broker controller?

Hi Kishore, Sorry for the long disappeared. There is no other errors regarding checkpoint file. This problem occured in our production enviroment, I'll try to reproduce them but still got no time to do it. I'll inform you if some thing come up. Thanks again! 2015-08-21 14:15 GMT+08:00 Kishore Se

Auto preferred leader elections cause data loss?

Hi group, I've been through some data loss on kafka cluster, one case maybe caused by the auto preferred leader election. Here is the situation: 3 brokers = {0,1,2}, 1 partition with 3 replicas on 0/1/2, all in sync while 0 is leader and 1 is controller, current offset is 100. And here is my hyp

Re: port already in use error when trying to add topic

Try throwing JMX_PORT=9998 In front of the command. Anything other than 9994 Foo On Friday, September 11, 2015, allen chan wrote: > Hi all, > > First time testing kafka with brand new cluster. > > Running into an issue that i do not understand. > > Server started up fine but I get error when