I wish I had a better answer for you but you can't safely and automatically have a setup across 2 datacenters where you can be guaranteed that the loss of one data center won't cause the cluster to go down. So, what you want to do, you cannot do. I wrote a bit about designing x-dc ZK clusters a while ago, not sure if any of it is relevant to you but you can read it here: http://www.elidedbranches.com/2012/12/building-global-highly-available.html
On Fri, Jun 3, 2016 at 3:55 PM, Nomar Morado <[email protected]> wrote: > Will be using ZK with Apache Kafka and don't know if I can get away of not > using ZK > > > > Printing e-mails wastes valuable natural resources. Please don't print > this message unless it is absolutely necessary. Thank you for thinking > green! > > Sent from my iPhone > > > On Jun 3, 2016, at 3:51 PM, Camille Fournier <[email protected]> wrote: > > > > You could put the remaining available node in read-only mode. You could > > reconfigure the cluster to have the majority nodes in the remaining data > > center, but it would require reconfiguration and restart of the nodes in > > the living data center. But there's no automatic fix for this, and if you > > can safely override the quorum rule for your application perhaps you > don't > > need to use ZK at all? > > > > On Fri, Jun 3, 2016 at 3:44 PM, Nomar Morado <[email protected]> > > wrote: > > > >> Thanks Shawn. > >> > >> Is there any settings to override the quorum rule? Would you know the > >> rationale behind it? > >> > >> Ideally, you will want to operate the application even if at least one > >> data center is up. > >> > >> Thanks. > >> > >> > >> Printing e-mails wastes valuable natural resources. Please don't print > >> this message unless it is absolutely necessary. Thank you for thinking > >> green! > >> > >> Sent from my iPhone > >> > >>>> On Jun 3, 2016, at 9:05 AM, Shawn Heisey <[email protected]> wrote: > >>>> > >>>> On 6/2/2016 4:06 PM, J316 Services wrote: > >>>> We have two data centers and got two servers at each. At an event of a > >>>> data center failure, with the quorum majority rule - the other > >>>> surviving data center seems to be no use at all and we'll be out of > >> luck. > >>> > >>> You are correct -- the scenario you've described is not fault tolerant. > >>> > >>> When setting up a geographically diverse zookeeper ensemble, there must > >>> be at least three locations, so if there's a complete power or network > >>> failure at one location, the other two can maintain quorum. One > >>> solution I saw discussed was a fifth tie-breaker server in a cloud > >>> service like Amazon EC2, or you could go full-scale with two more > >>> servers at a third datacenter. > >>> > >>> Thanks, > >>> Shawn > >> >
