Re: Kafka failover with multiple data centers

Hi Soumitra, We're working on that. The Idea here is to use Kafka to get brokers' information of the topic and use Kafka client to find coresponding offsets on new cluster ( https://jeqo.github.io/post/2017-01-31-kafka-rewind-consumers-offset/). You need kafka >=0.10.1.0 because it supports

Re: Kafka failover with multiple data centers

Hi, did you guys figure it out? Thanks Soumitra On Sun, Mar 5, 2017 at 9:51 PM nguyen duc Tuan wrote: > Hi everyone, > We are deploying kafka cluster for ingesting streaming data. But > sometimes, some of nodes on the cluster have troubles (node dies, kafka > daemon is

Kafka failover with multiple data centers

Hi everyone, We are deploying kafka cluster for ingesting streaming data. But sometimes, some of nodes on the cluster have troubles (node dies, kafka daemon is killed...). However, Recovering data in Kafka can be very slow. It takes serveral hours to recover from disaster. I saw a slide here