Re: Kafka-streams process stopped processing messages

Hi Shimi, Could you provide more info on your setup? How many kafka streams processes do you have and from how many partitions are they consuming from. If you have more processes than partitions some of the processes will be idle and won’t do anything. Eno > On Apr 30, 2017, at 5:58 PM, Shimi

Re: Kafka-streams process stopped processing messages

Hi Eno, I am afraid I played too much with the configuration to make this productive investigation :( This is a QA environment which includes 2 kafka instances and 3 zookeeper instances in AWS. There are only 3 partition for this topic. Kafka broker and kafka-stream are version 0.10.1.1 Our kafka-

Re: Kafka-streams process stopped processing messages

Hi Shimi, 0.10.2.1 contains a number of fixes that should make the out of box experience better, including resiliency under broker failures and better exception handling. If you ever get back to it, and if the problem happens again, please do send us the logs and we'll happily have a look. Tha

Replication quota only during partition movement?

Hi, I have been reading about replication quota here . I just wanted to know does the replication quota come into play only when kafka-reassign-partitions.sh script is called? Is there any replication quota app

kafka zookeeper integration with storm

Hey guys , I am unable to use kafka zookeeper with strom integration. I am passing kafka bin dir location in zookeeper root for storm but its throwing error. Keep learning keep moving .

kafka consumers as kubernetes pods - best practice

Im looking for suggestions as to how to manage k-consumers when they are run as kubernetes pods - especially in an auto-scaling environment. Looking at the output of our logging it seems like we're dropping data when a pod is moved between hosts despite doing (what I believe is) an orderly shutdown