What is the latency between the two datacenters? I ask because unless they are very close, you probably don’t want to do any form of synchronous replication. The Confluent Replicator (coming very soon in Confluent Enterprise 3.1) will do async replication of both messages and configuration metadata between datacenters. It’s still up to you to monitor for what your app considers acceptable lag between the two datacenters but at least now that is possible using timestamps and the new offsetsForTimes() capability added in 0.10.1. Accurate message timestamp based offset lookup is necessary because the offset numbers for a given message will not match in both datacenters.
-hans > On Oct 28, 2016, at 8:08 AM, Mudit Agarwal <mudit...@yahoo.com.INVALID> wrote: > > Thanks dave. > Any ways for how we can achieve HA/Failover in kafka across two DC? > Thanks,Mudit > > From: "Tauzell, Dave" <dave.tauz...@surescripts.com> > To: "users@kafka.apache.org" <users@kafka.apache.org>; Mudit Agarwal > <mudit...@yahoo.com> > Sent: Friday, 28 October 2016 4:02 PM > Subject: RE: Kafka Multi DataCenter HA/Failover > >>> without any lag > > You are going to have some lag at some point between datacenters. > > I haven't used this but from taking to them they are working or have created > a replacement for MirrorMaker using the Connect framework which will fix a > number of MirrorMaker issues. I haven't talked to anybody about Kafka > failoer. > > -Dave > > -----Original Message----- > From: Mudit Agarwal [mailto:mudit...@yahoo.com.INVALID] > Sent: Friday, October 28, 2016 9:38 AM > To: Users > Subject: Kafka Multi DataCenter HA/Failover > > Hi, > I learned that Confluent Enterprise provides Multi DC failover and HA > synchronously and without any lag.I'm looking to learn further information > and more detailed documentation on this.I have gone thorugh the white paper > and it just talks about Replicator. > Any pointers for more information will be helpful. > Thanks,Mudit > This e-mail and any files transmitted with it are confidential, may contain > sensitive information, and are intended solely for the use of the individual > or entity to whom they are addressed. If you have received this e-mail in > error, please notify the sender by reply e-mail immediately and destroy all > copies of the e-mail and any attachments. > >
