Detecting cluster down in consumer

2019-11-11 Thread Sachin Kale
Hi, We are working on a prototype where we write to two Kafka cluster (primary-secondary) and read from one of them (based on which one is primary) to increase the availability. There is a flag which is used to determine which cluster is primary and other becomes secondary. On detecting primary cl

Re: Detecting cluster down in consumer

2019-11-11 Thread M. Manna
Hi, On Mon, 11 Nov 2019 at 11:55, Sachin Kale wrote: > Hi, > > We are working on a prototype where we write to two Kafka cluster > (primary-secondary) and read from one of them (based on which one is > primary) to increase the availability. There is a flag which is used to > determine which clus

Re: Detecting cluster down in consumer

2019-11-11 Thread Ryanne Dolan
Sachin, assuming you are using something like MM2, I recommend the following approaches: 1) have an external system monitor the clusters and trigger a failover by terminating the existing consumer group and launching a replacement. This can be done manually or can be automated if your infrastructu